What is the difference between an AB Test and a Multivariate Test?

A/B Tests and Multivariate tests both allow you to optimize your boxes to increase conversions.

So what are the differences between an A/B test and a multivariate test in Digioh?

A/B Tests:

  • You can test design or copy variations. (You do this by starting from one lightbox and creating variations).
  • You can see results in near-real-time on the A/B Tests page. We can integrate with your analytics system so you can see how A/B tests impact pageviews, revenue, etc.
  • After the test finishes naturally (end date or max impressions), Digioh automaticallys display the winner of the A/B Test going forward.
  • If you stop the test manually, Digioh will display the “primary” lightbox, which by default is the original lightbox. If you want to display the variation, you can make it the “primary” lightbox.
  • You can run any number of tests at the same time, but in each test only 2 versions (i.e. A vs B).

Multivariate Tests:

  • You can test conditions, widget type, and copy/color. For example, you can test exit-intent lightbox vs. 3-second timer slider vs. static inline form. (For multivariate tests, you must create different lightboxess, not just a variation).
  • You can run any number of box versions against each other (A/B testing is limited to 2).
  • Important to note: It may take a long time to get results if conditions are very different. For example, the 3-second slider will likely show up more often than an exit-intent lightbox. Therefore, it will stop displaying the 3-second timer once it hits the impression target.
  • You can pick a traffic percentage. This is helpful when you have something that already works. You keep that going, duplicate it, then use the duplicated version in a multivariate test for a small portion of your traffic, such as 30%.
  • Digioh can integrate with your analytics system to help you see if your multivariate tests impact pageviews, revenue, etc.