Digioh provides powerful functionality and analytics that allows you to optimize design, content, and targeting conditions that maximize your conversion metrics. You can run Experiments with up to 10 different boxes, and your site visitors will be randomly bucketed so that they are only eligible to see one of the boxes. The test bucket is sticky across sessions, so your visitors will get a consistent experience.

Setting up an Experiment

Click on Experiments in the top navbar, you’ll see an empty list of Experiments. Click on New Experiment.

Let’s walk through the available configuration options:

1. Experiment name: choose a name that is brief and for purpose. You’ll be able to see all of the configuration options later, so don’t feel that you need to add a lot of detail.
2. Overall Traffic Percent: with this option, you can have a “hold out” group that will not see any Boxes in the Experiment, or accept the default 100%. You might want to have a large holdout group if you are concerned that the Boxes might harm your site conversion.
3. Automatically Start/Finish: this optional feature allows to set a future start date, and end date, or a maximum number of Box impressions for the Experiment. When an Experiment is automatically finished, the winning Box will be chosen based on the conversion rate of either Submissions/Impressions or Redirects/Impressions (whichever is larger). While this feature can be useful, it may not be suitable for all situations, e.g. if your Boxes have multiple submissions. When in doubt, leave this option off and review detailed Digioh analytics for the Experiment to choose a winner manually.
4. Box N: Here you choose which Boxes to include in the Experiment. Boxes can only be in one Experiment at a time, and Experiments cannot start until all Boxes are turned on and Published.
5. Traffic %: This is the amount of enabled overall traffic that Digioh will randomly distribute as eligibility for each Box. An equal split between Boxes will generate statistically significant results the fastest, but if you are concerned about potential negative impact from one or more of the Boxes you can reduce their percentage. For example, you may have an already live box that you want to run against a new “challenger”, with low traffic over a longer period.

Testing Tips

Typically, before configuring an experiment, you will duplicate an existing Box from an already live Box on site (do this under the actions menu in the Box list). Then you can make changes to the duplicate. When running Experiments, it’s important to test one change at a time. If you change multiple aspects of the design and conditions between Boxes, you won’t know what is causing a positive (or negative) effect. Or worse, you may have one change that is positive, but is negated by another change, and so appears like neither change had any effect.

Decide what metric(s) you want to optimize, and find them in Digioh Analytics. Digioh does not dictate what a Box “conversion” is, you’ll need to decide that based on what’s important to your business. It may be email collected, redirects to another page on your site, quiz completions, or some other metric.

While you can test different conditions with Experiments, there is some nuance to this. Generally, when testing conditions changes, you should care more about the total number of conversions vs. the conversion rate. As an extreme example, if you change conditions so that a particular Box receives only 20% of the impressions vs. other Boxes in the test, then that Box will have to perform at 5X the conversion rate to have an equal effect on your business.

Managing Experiments

On the Experiments page, you’ll see a list of all Experiments ever created.

Use the down chevron control on the right to expand Box details:

This shows configuration and basic metrics for each Box in the Experiment, links to compare Boxes in more detailed Digioh Analytics, and a control to declare a Box as the winner. Note that when you select a Box as winner, then all other Boxes will be turned off. Otherwise, potentially all of the Boxes could be shown to your visitors at the same time!

When Experiments finish automatically, Digioh will start showing the winning Box to all visitors. You can override this decision simply by turning off Boxes other than the one you choose.

Have any questions about A/B or multivariate testing? Send us an email, and we’ll be glad to help!