What is the difference between an AB Test and a Multivariate Test?

AB Tests and Multivariate tests allow you to test your way to higher sign ups and sales.

So what are the differences between an AB test and a multivariate test?

AB Tests:

  • You can test design or copy variations. (You do this by starting from one lightbox and creating variations).
  • You can see results in near-real-time on the A/B Tests page, and we can also integrate with your analytics system so that you can see how your AB tests have other impacts (pageviews, revenue, etc).
  • After the test finishes naturally (end date or max impressions), Digioh will automatically display the winner of the AB Test going forward.
  • If you stop the test manually, Digioh will display the “primary” lightbox, which by default is the original lightbox. If you want to display the variation, you can make it the “primary” lightbox.
  • You can run any number of tests at the same time, but in each test only 2 versions (i.e. A vs B)

Multivariate Tests:

  • You can test conditions, widget type, and copy/color. For example, you can test exit intent lightbox vs. 3 second timer slider vs. static inline form. (For multivariate tests you need to create different lightboxes, not just a variation).
  • You can run any number of box versions against each other (compared to AB testing which is limited to 2).
  • Important to note: It might take a long time to get results if the conditions are very different. For example, the 3 second slider will likely show up more often than an exit intent lightbox and therefore it will stop displaying the 3 second timer once it hits the impression target.
  • You can pick a traffic percentage. This is helpful when you have something that’s already working. You keep that going, duplicate it, then use that duplicated version in a multivariate test for only 30% of your traffic. This way only a small portion of your traffic sees the test.
  • Digioh can integrate with your analytics system so that you can see if your Multivariate tests have other impacts (pageviews, revenue, etc).