What’s the best A/B Testing Tool?
I have implemented and tested a wide range of A/B testing tools over the past few years with businesses of all sizes and types. The best can be different based on your goals, business/team size and whether you want to test on the web or mobile.
My goal is to give you the right information and share my experience so that you can make the best decision for what A/B testing tool is best for you.
There are many elements to an A/B testing platform, but many of the functions overlap when compared side by side. Here are the A/B Testing Tools we recommend the most:
Google Optimize A/B Testing Tool
Google Optimize integrates with your Google Analytics account to improve site experiences for your users. Using both tools together can root out problem areas of your site and turn those insights into actionable changes. This is our preferred Enterprise solution with the GA360 Suite.
Google Optimize
Free & Enterprise-
Integrates with GA
-
Simple Tag Setup
-
Scales to Enterprise
VWO A/B Testing Platform
VWO was built as an A/B testing and conversion optimization platform explicitly made for brands. It uses a drag-and-drop interface to create A/B, multivariate quickly, and split URL tests. It is one of the only A/B test tools to use Bayesian statistics to give you faster results.
VWO
Affordable & Easy-
Multivariate Support
-
Heatmaps & Recordings
-
Easy Setup
AB Tasty
AB Tasty is best suited tool for medium-sized businesses that want to leverage a visual interface to run A/B and multivariate tests. You can also run tests based on weather, geolocation, and more.
AB Tasty
Mid-Sized Businesses-
Optimization Features
-
Enterprise Features
-
Hands On Support
Optimizely
Optimizely boasts having 24 of the Fortune 100 companies using its tools. It is specifically built for marketing, engineering, and product teams and has powerful web testing abilities for testing cookies, dynamic pages, devices, and browsers.
Optimizely
Pricey, Enterprise Only-
Strong Technology
-
Enterprise Features
-
Contracts Only
Crazy Egg
Crazy Egg is a great alternative for smaller businesses and startups. Like VWO it just requires a single snippet and can accomplish your test setups using their tool, they also offer user insights like heat-maps & user recordings.
Crazy Egg
Startup Friendly-
Easy Setup
-
Heatmaps & Recordings
-
Free Trial
Convert
Convert is best suited for small businesses but has many features typically reserved for large companies like click-through rate variation and measuring ROI. It has a ton of integrations with 3rd party tools like Shopify, WordPress, and HubSpot.
Convert
Well Built, Great Support-
Fast Technology
-
Higher Test Confidence
-
Free Trial & Support
How do A/B testing tools work?
Before delving into how A/B testing tools work, it is essential to state that not all A/B tools will fit for every type of business. All of the tools I mentioned above I provided some context for who each tool is best suited, this is important as not every size fits all.
Before starting an A/B test, a business will come up with a problem they want to solve. Then the business stake-holders formulate a hypothesis about what solution could help solve the problem.
A new landing page heading, a new product photo, or a different buy button are examples of variables. Let’s take a sales page “Buy Now” button, for example. The business will build two identical pages; one with button A, and one with button B. Web traffic is run to both pages at equal amounts.
The winner is determined by looking at which buy button converted the best when the same amount of traffic was run on both pages.
The hypothesis of “Can changing our Buy Now button increase conversions?” was then confirmed using data collected from your A/B testing tool.
How long should you run an A/B test?
Using an A/B testing tool can be a controversial topic. If your business tests the wrong variables, or even worse a false positive, where your testing platform says it was a success, but reality was worse.
When done right, an A/B testing tool can help optimize conversions and give you a way to scale up faster. While it may sound like every business should use A/B testing tools, some companies are not ready to use these tools.
If your business does not generate at least 1,000 conversions per month, you are not ready for an A/B testing tool. If you are at 1,000 or more conversions per month, it is time to think about what conversion tool is right for your business.
You should run your A/B test long enough to ensure your test hypothesis is either confirmed true or false. Not running your A/B tests long enough is one of the main reasons why A/B tests don’t work.
The industry standard is that A/B tests should run until your hypothesis is 95-99% proven right or wrong based on data.
Summary of A/B Testing Implementations
- Have a hypothesis.
- Have a goal you are measuring.
- Select the right tool.
- Test something big enough to get a read.
- Less traffic, test bigger things.
- Let the test run long enough to have confidence.