Debunking these myths will help you make more data-driven decisions that are accurate and effective. When done correctly, A/B testing has been shown to generate up to 30-40% more leads for B2B sites, and 20-25% more leads for e commerce sites.
Myth # 1: Every change warrants A/B testing
As a general rule, anything that doesn’t alter SERP doesn’t need to be A/B tested. This means that you don’t need to test two headlines that simply switch out punctuation or “it” for “the” since they won’t change your conversion rates.
You will need to test for things like the colour of your call to action text or headline position though; basically, anything that can affect the potential client’s affinity to become a conversion should always be tested.
Myth # 2: A/B testing costs too much
It seems that far too many internet marketers somehow bought into the concept that A/B testing was only for larger, tech-savvy firms.
While that might have been true at one point, free tools like Google Analyitics’ Content Experiments are changing that scenario. And while that still might be a little tech-heavy for some, there are many cheap tools that are easy to use that will allow you to get statistically significant results. Plus, the more math you can do on your own, the cheaper your overhead for testing becomes (since you eliminate the costlier tools).
Myth # 3: A/B testing is bad for SEO
Let’s just state this right upfront: A/B testing has no bearing on your SEO since it is not seen as duplicate content in Google’s algorithm (which would warrant SERP penalties). On the contrary, Google provides guidelines and endorses A/B testing in order to bring more conversions and traffic to your site.
Myth # 4: Stop running the test if one treatment is the immediate winner
The worst thing you can do when running A/B treatment tests is make a premature call. You need to understand that you have to wait for a 95% level of confidence in your testing for the results to be statistically significant.
Use a confidence interval to help you not jump the gun and even if there’s a landslide right up until the very end, keep your test running until your sample size and time frame become statistically significant.
Myth #5: The winning treatment will look the best
One thing that’s hard to grasp about A/B testing is that the treatment that performs the best may be you’re least favourite one, it might be downright hideous. But who cares, it’s turning conversions and that’s what counts.
Myth # 6: The A/B test is your final step.
Testing with A/B treatments is a continual process that should always be split up to optimize your site bit by bit. For instance, finding out a larger font CTA works over a smaller font is great, but now try that larger font in different areas or colours.