Inconstant periodicity

[bitheerani90]

One of the classic mistakes made when analyzing A/B test results is the frequency of the activities. For example, it is no use having segmentation, correct sampling, high reliability, well-defined objectives and hypotheses, but carrying out tests netherlands mobile database random times. Or worse, carrying out a single test and thinking that it will be valid for all eternity!

You should wait at least 7 days between tests to see how traffic changes between weekdays and weekends. However, it’s best to wait at least 2 weeks to avoid holiday or seasonal impacts. You should also expect to get at least 100 conversions to get reliable data.

Have the right days to start and end your tests, consider seasonality and special dates. Every external traffic factor or biased behavior can harm your entire result.

Tips to avoid falling into these traps

Now that you know what the most common mistakes are, we have put together some tips that will help you avoid making the same mistakes and help your online business grow even more! Here are some tips:

Carefully define the right metric for testing based on properly structured objectives;
Build clear hypotheses: Don’t make assumptions about what customers want.
Calculate the sample size (number of visitors) before starting the test.
Wait until the calculated sample size is reached before stopping the test — do not stop the test prematurely;
Adjust the confidence level when performing post-test segmentation or evaluating more than one alternative;
Decide on a confidence level before the test begins and stick to that threshold when evaluating results after the test is complete.
Good results require time and analysis. These are “just” a few mistakes that our team has selected to help you stop working based on guesswork! It is important to ensure a strategy and a guiding principle. If you want to know more about this subject, read this article .

Always remember: it's not just in the conversion rate that you find the secret to your (in)success, but in all the