Glossary

Bandit Testing

When the experiment is done using multivariate or A/B/N testing methodology, additional variation is using bandit testing. Bandit testing aims to improve the effects of the testing process by eliminating losing variations from the test as it progresses.

This is done using various algorithms that track the performance of variations in real time and adjust the traffic allocation favoring the winning ones. The danger lies in eliminating losing variations too early, while the primary benefit is increasing the revenue during the testing process.

Bandit testing is most frequently used to test various landing pages that are used for a limited amount of time, for example for special discounts during the holidays or promotional campaigns.