Placement A/B Test

Placement A/B Testing allows you to experiment with different configurations within your Offer Wall placements to help you discover the most effective settings for boosting user engagement and revenue.

With Placement A/B Testing, you can make data-driven decisions to enhance your Offer Wall's performance. Test variations in exchange rates, colors, banners, and layout to gain insights into what resonates best with your audience. This feature empowers you to strike the perfect balance between user satisfaction and profitability, ultimately driving your Offer Wall's success.

Creating an A/B Test Experiment

To begin running a test, follow these steps:

  1. Login to the DT Console here
  2. Select the App on which you want to run the test

    Screen Shot 2023-08-23 at 10.56.55.png

  3. Select the Placement for which you want to run the test

    Screen Shot 2023-08-23 at 11.00.08.png 

    The Placement Test window opens.
  4. Click Create Experiment
    The test options window opens displaying all the options

    Screen Shot 2023-08-30 at 14.30.34.png

  5. Provide a name for the test
  6. Click Add Variants to add the variants that you want (up to 4 variants is possible, baseline + 3 options)
  7. Make all of the configurations you would wish to test, exchange rates, banners, colors, etc.
    Make sure you make changes to the variant and not to the Baseline.
  8. If you want to test the exchange rates, enter the exchange rate and the countries for which you want to run the experiment.Screen Shot 2023-08-23 at 11.09.37.png
  9. Click Add Entry to add additional exchange rates and countries
  10. In case you want to test any UI customization such as colors or banner images you can set it in the UI Customization section.

    Screen Shot 2023-08-30 at 14.32.33.png

  11. Once you are finished configuring the variants of your experiment, click Start Experiment.

    Note: During the experiment, you can change the percentage between the variants and configurations, but mind that it might have an impact on the significance of the results.

Monitor Test Results

To monitor the performance of your test, follow these steps:

  1. From the DT Console on the left toolbar click Dynamic Reports -> Offer Wall Report
  2. From Dimensions select Placement Experiment and Variant

    Screen Shot 2023-08-23 at 11.30.22.png

  3. Mark the name that you gave to your experiment
  4. Switching to 'Performance View' can make your analysis more accurate and review the results attributed to the moment of the changes.
  5. You can monitor your KPIs for the experiment.
    Some relevant metrics that might be relevant to include in the report would be:
    Metric Description
    OFW Unique Users To verify the distribution between the variants
    Publisher Revenue Review overall revenue coming from each variant
    ARPDEU The average revenue per engaged user for each variant
    Container CTR The number of users who entered Offer Wall clicked on at least one offer
    Offer CTR The number of users who started offers
    Click/Conversion Rate The overall conversion rate for the variant
  6. You can monitor the aggregated results of the experiment by looking at the table view

  7. You can also monitor the results over time by adding the date to the Dimension Split and looking at the line chart

Choosing the Winning Variant

Once you have completed the experiment, examined the results, and decided which is the best variant to use, do the following steps on the Placement Experiment window:

  1. From the list of variants in your experiment, click the variant you want
  2. Click End Experiment 

    Screen Shot 2023-08-30 at 14.33.49.png  

  3. The winning variant is then set as the default placement configuration in your placement.

Back to Top ⇧