Skip to main content

A/B Test

1. Functions​

  • Randomly group users to implement different Ad realizing strategies, and then compare the data of different groups to select the strategy plan with better effect.

  • Support app dimension to set up A/B test, one A/B test can select multiple TradPlus Ad units at the same time, and up to 10 test groups can be created.

  • Support viewing A/B test data from app dimension and Ad dimension, as well as viewing historical ended A/B test group data.

  • You can select the optimal test group after the A/B test is finished with one click.


2. Usage scenes​

According to the single-variable principle, we do A/B testing by cutting a small portion of traffic to verify the effect of realizing different Ad strategies.


3. Usage guides​

1）Click【Advanced Settings】-【A/B Test】


2）Select the app that needs A/B testing in the app list on the left and click [Create A/B Test].


3）Configure A/B testing, with configuration item descriptions in order:

a. App name：Select the apps that need to be A/B tested.

b. Ad Unit：Check the units that need to be A/B tested. One or more Ad units can be used for testing.


c. A/B group：

  • Traffic ratios can be adjusted by adding or subtracting weights.


  • Check “Duplicate existing Waterfall” to automatically copy the waterfall from control group A to experimental group B.


  • Click Add Group to add multiple experimental groups.


d. Start Time：choose【effective immediately】or【specified time】

e. Name of test：Enter a customized test name.

f. Description of test：Remarks, if any, can be filled in here.


4）After confirming that the configuration information is correct, click Save.


  1. Configure control group/experimental group waterfall

  • The original waterfall of the unit will be automatically synchronized to the “A control group”.

  • Group B waterfalls need to be configured manually and currently offer two configuration portals:

i. After saving the A/B test, click [Go to Configuration] to jump to the “Segments” page to configure, or directly click [Segments] to enter the page to start configuration.


ii. Click [Overview] - [Segments] in the upper right corner of the A/B test page to jump to the “Segments” page for configuration.


  • Return to the segments page and use the “Duplicate A/B Test Group” function to quickly configure waterfalls for other segments.

    • This function will duplicate all the configurations of the waterfall currently used by a group to the specified group (including Plancement ID, Sorting, Cache Count, Bidding Base Price, etc.).

    • After copying, the original configuration under the specified group will be deleted.

For example, if you duplicate the waterfall of group A to group B, then: group B will use the configuration of group A; the original configuration of group B will be deleted; the current configuration of group A will remain.


5）After completing the test, you need to click [End Test] and choose whether to use the optimal grouping strategy. You can choose to apply it to all Ad units, or choose to apply it to specific ad units.


4. FAQs​

Q：What is the component logic of A/B tests?​

A：

  • After turning on A/B tests, the first time a user requests an Ad during the A/B test cycle, it will go to the hit group based on the traffic allocation ratio.

    • For example, if the traffic allocation ratio for group AB is 50% each, the user will be randomly assigned to either group A or group B. Finally, group A and group B are each assigned 50% of the users.

    • Once a user hits a grouping, it will not change and will keep hitting that grouping subsequently. That is, once a user has gone to group A, he will not go to group B.

  • If you adjust the group A/B volume policy, for example, adjusting the traffic allocation ratio from 50% each to 90% for group A and 10% for group B, then the users will be reassigned. That is, a portion of the users from group B are randomly assigned to group A; 50% of the original users in group A still go to group A.


Q：How can I view data from A/B tests?​

A：On the A/B test page, view the A/B test data by Ad unit dimension. (Note: All data on the A/B test page is TradPlus buried data, not Networks data)


If the A/B test is closed, you can view the data of the A/B test that has been configured in history in the A/B test page [History].


  • If each subgroup uses a different Ad unit ID, in each subgroup you can view the Networks data for the Ad unit ID used by the current subgroup.
  • If each subgroup uses the same Ad unit ID, the Networks data for the same Ad unit ID will be the same in each subgroup since the Networks data can only be split by Country dimension. You can choose to view the TradPlus buried data under the segments dimension to see how the data is performing for that subgroup.

Q：How to deal with the data generated by the original waterfall after the A/B test takes effect?​

A：According to the effective time is divided into two cases:

  1. If the configured effective time is "Effective immediately ” :

    • The data generated by the history caching strategy (not the strategy configured in the current A/B test) will be categorized into the history traffic group. Ad sources for historical traffic grouping cannot be configured at this time.


  1. If the effective time of the configuration is “specified” time to take effect :

    • The current waterfall configuration will be categorized into the “current group”, and the data generated before the A/B test takes effect will be categorized into the current group;

    • Until the specified time when the A/B test takes effect, the “current grouping” will be replaced by the “historical traffic” grouping.