Multivariate Test overview

Multivariate Testing (MVT) in Adobe Target compares combinations of offers in elements on a page to determine which combination performs the best for a specific audience, and identifies which element most impacts the activity’s success.

MVT overview

Multivariate testing can help you discover the relative influence specific elements have on conversion, compared to other elements on the page. It can also help you to refine a combination of elements that have been shown to be effective.

One advantage a multivariate test provides compared to an A/B test is the ability to show you which elements on your page have the greatest influence on conversion. This is also known as the “main effect.” This information is useful, for example, by helping you determine where to place content that you want to receive the most attention.

Multivariate tests also help you find compound effects between two or more elements on a page. For example, a particular ad might produce more conversions when combined with a certain banner or hero image. This is also known as the “interaction effect.”

Adobe Target uses full-factorial multivariate tests to help you optimize your content. A full-factorial multivariate test tests all of the possible combinations of content with equal probability. For example, if you have two page elements with three offers each, there are nine possible combinations (3x3). Three elements, with two containing three possible offers and one with two offers, provide 18 options (3x3x2).

In Adobe Target, each combination is one experience. The multivariate test compares each experience so you can learn which combinations are the most successful. At the same time, data is collected and analyzed to understand how each location and the offers influence the success metric.

Because of the number of combinations that can be generated, a multivariate test requires more time and traffic than an A/B test. The page must receive enough traffic to produce statistically significant results for each experience. To obtain useful results, you need to understand the amount of traffic your page receives and test the optimal number of combinations for the right amount of time to get the required results. Adobe Target’s Traffic Estimator can help you design a test that works with your traffic. Before you use the Traffic Estimator, you should have good statistics showing the number of impressions and conversions your site normally receives. Consider your traffic levels per day. The more experiences in an activity, the more traffic the activity will need to include or the longer your activity will need to run. If your traffic isn’t very high, you should test a small number of combinations; otherwise, the amount of time required to produce meaningful test results might be too long to be useful.

When to use MVT vs A/B

Multivariate tests can be used together with A/B tests to optimize your page. Examples of when you might want to use them together include:

• Use an A/B test to optimize your page layout, followed by an MVT test to determine the best content in each element on the page.

  An A/B test can provide important feedback on the layout, and MVT tests excel on testing the content within the elements in your page design. Running an A/B test on the layout before testing multiple content options can help you determine the best layout and the most impactful content.

• Use an MVT test to determine which element is the most important, then follow up with a more focused A/B test on that element.

  When the number of different experiences exceeds five and spans two or more elements, it’s a good idea to consider an MVT test before running your A/B tests. The MVT test shows which areas on the page are most likely to improve conversion. These are the elements that a marketer should focus on. For example, the MVT test might show that the call to action is the most important element for meeting your goals. Once you have determine which elements and content are most useful for helping you meet your goals, you can run an A/B test to further refine the results, such as to test two specific images against each other, or comparing the wording or colors of a call to action. By following an MVT test with one or more A/B tests, you can determine the best possible content for the results you desire.