Bid Experiment Results And Rule Implementation

Once your bid experiment has run to completion it is time to analyze and act on the results. This page explores the Bid Experiments results dashboard and how to create a rule for successful tests. For a full outline on how to set up and launch a bid experiment, please reference our bid experiment setup guide.

Analyzing Your Bid Experiment Results

After a Bid Experiment runs to completion or is manually stopped by a user, your Bid Experiment tab will produce the results of the campaign, which can be found at Bid Experiment > Dashboard. The results displayed in this dashboard should be used to help you decide which of the following actions to take:

1. Declare a successful experiment and implement the experiment settings as a rule
2. Declare an unsuccessful experiment and retain your current bid settings
3. Declare inconclusive results and re-run the experiment with a larger sample size

Using The Bid Experiment Dashboard

Your Bid Experiment Dashboard displays several charts that break down the performances of Version A (your current settings) and Version B (your bid adjustment settings). While we encourage you to closely examine each chart, it is important to focus on the Experiment Results and Explanation widgets for each experiment you run.

The Explanation widget breaks down the performance of Version A (your current settings) and Version B (your adjusted bid settings) based on their average profits per 1000 pings. Furthermore, this widget explains the confidence level of the experiment to help you decide whether or not you should trust the results of the experiment.

Based on the results displayed in your experiment dashboard, you should be able to determine exactly how to move forward. If you still aren’t sure how to act on the results of your experiment, please read our two-part bid experiment case study and results analysis.

Bid Experiment Rules

Bid experiment rules allow you to immediately implement the settings of your bid experiment to all future pings that meet the bid experiment’s criteria. In other words, once a bid experiment runs to completion and you determine that you would like to implement the changes, you can create a rule that applies your bid experiment criteria to all future pings instead of implementing these changes by making manual bid and filter set adjustments in the boberdoo system.

When Do We Suggest Creating Rules From Your Bid Experiments?

1. The results of the experiment proved Version B to be a better option either considering the Buying Rate or the Average Profit Per Ping as the main performance indicator and the confidence level of the experiment is above 95%. (You may still want to implement a rule if your confidence level falls between 80%-95% if the additional experiment result charts and analysis indicate a reduced risk of uncertainty.)
2. You want to save time setting up new filters or modifying existing ones that match the consumer segment defined in the experiment.
3. Your profit margin is set on the SRC side (instead of the filter set side) and cannot define separate profit margins for product categories or consumer segments.

How To Set Up Bid Experiment Rules

Implementing a Bid Experiment Rule is simple.

Option 1: Open the Dashboard for the bid experiment you’d like to implement a rule for. From the Explanation widget, click Create a Rule.

Click OK to confirm the rule creation.

Option 2: Navigate to the Experiments tab and click the tool titled Make Rule From Experiment.

Click OK to confirm the rule creation.

Once your rule is created, you will be redirected to the Rule Management subtab. At this point your Rule is Inactive. To apply the selected experiment’s changes to future pings, you need to activate the rule by clicking the Activate Rule tool.

Once activated, the settings for your selected bid experiment will now apply to all qualifying pings. This rule can be deactivated and deleted at any time.