{"id":47028,"date":"2019-11-12T09:52:45","date_gmt":"2019-11-12T14:52:45","guid":{"rendered":"https:\/\/www.sogolytics.com\/blog\/?p=47028"},"modified":"2019-11-12T09:53:43","modified_gmt":"2019-11-12T14:53:43","slug":"why-a-b-testing-is-better-than-a-then-b-guessing","status":"publish","type":"post","link":"https:\/\/www.sogolytics.com\/blog\/why-a-b-testing-is-better-than-a-then-b-guessing\/","title":{"rendered":"Why A\/B testing is better than \u201cA then B\u201d guessing"},"content":{"rendered":"

In recent years, data has dethroned oil and become the <\/span>world\u2019s most valuable resource<\/span>.<\/span><\/a><\/p>\n

<\/div>\n

Most companies understand the importance of data, soliciting customer feedback<\/a> and utilizing numerous tools for analytics. Many also appear to recognize the value of experimentation as a source for great insights. However, when it comes to executing smart business experiments, they prove that it\u2019s easier said than done.<\/span><\/p>\n

<\/div>\n

Beyond a standalone product or website survey, A\/B testing is one of the most popular ways to gather data, particularly in online settings. Many companies use A\/B testing to measure the efficiency of two versions of a webpage expected to convert visitors into customers, like a landing page for a product. According to Invesp, <\/span>71 percent of companies run two or more A\/B tests a month<\/span><\/a>.<\/span><\/p>\n

<\/div>\n

But what exactly is an A\/B test, why is it important, and what advantages does it offer over the guessing methods many people still use?<\/span><\/p>\n

<\/div>\n

What is A\/B testing?<\/b><\/h2>\n

A\/B testing is controlled experimentation that allows you to simultaneously compare \u201clike for like\u201d scenarios where one version has a different element. \u2018Bucket\u2019, \u2018split\u2019, or \u2018split-run\u2019 tests are other names for this type of experiment.<\/span><\/p>\n

<\/div>\n

A\/B tests involve the use of a randomly selected \u201ccontrol group\u201d and a \u201ctreatment group.\u201d The control group represents the experiment\u2019s comparison standard. You use this set of individuals to establish the baseline measure.<\/span><\/p>\n

<\/div>\n

In the context of webpage testing, these are the visitors you send to your current version of a given page, one with a conversion rate you\u2019re looking to improve.<\/span><\/p>\n

<\/div>\n

The treatment group, on the other hand, are the recipients of the experiment\u2019s varied element. These visitors will instead be directed to the new version of the same page which you\u2019ve come up with to test. It might have a different offer, color scheme, or call to action.<\/span><\/p>\n

<\/div>\n

An A\/B testing tool then measures each group\u2019s engagement with the page to find out which version is the most popular. In short, the tool will tell you which page is more successful at converting leads into customers.<\/span><\/p>\n

<\/div>\n

Since you\u2019ve only changed one aspect of the page, you can reasonably assume that the change is responsible for any differences in conversion rates, as long as you\u2019ve had a reasonable amount of traffic to each version.<\/span><\/p>\n

<\/div>\n

As well as webpages, A\/B testing can be run for emails, multimedia marketing strategies, paid internet advertising, and newsletters — nearly any step in the customer journey<\/a>. Web design is the most popular form, though, offering so many ways to control visitors’ experience and such rich opportunities to increase traffic and conversion and lower bounce rates.<\/span><\/p>\n

<\/div>\n
\"\"<\/div>\n
<\/div>\n

What is the alternative to A\/B testing?<\/b><\/h2>\n

Ignoring the benefits of A\/B testing, many companies still rely on what we\u2019ve termed \u201cA then B guessing\u201d. This is when a new version of a page is created, is put online, and then its performance is measured without concurrent testing of the old version.<\/span><\/p>\n

<\/div>\n

Imagine for a moment that you\u2019re thinking of changing the background color of your landing page to boost conversions. To evaluate the effect of this change, you create two versions of the same landing page. One version is the original page with the present color. The other is the new selection.<\/span><\/p>\n

<\/div>\n

Here\u2019s what happens next if you\u2019re following A\/B test best practices. You\u2019ll split your website traffic randomly and equally. One set of visitors will see the original version while the other views the modified version.<\/span><\/p>\n

<\/div>\n

Now imagine making that same change but doing the following instead: instead of splitting your audience, you deploy the updated version to your entire audience.<\/span><\/p>\n

<\/div>\n

How would you know whether any variations reported are the result of the color change? This method includes far too many variables to be successful. What if an increase in traffic is down to seasonality of your site, or a new ad campaign you started, or some recent press? By evaluating two versions simultaneously, you negate these effects.<\/span><\/p>\n

<\/div>\n

What about A\/A testing?<\/b><\/h2>\n

We should briefly mention another form of testing: A\/A testing. A\/A testing is similar in some ways to A\/B testing. Both experiments call for the use of a \u201ccontrol group\u201d and a \u201ctreatment group.\u201d However, an A\/A test has one significant difference: there is no disparity between the two testing scenarios. In essence, you\u2019re testing the same thing against itself.<\/span><\/p>\n

<\/div>\n

Validating the accuracy of A\/B testing tools is the main reason why people run an A\/A test. This type of analysis can also help to determine the baseline conversion rate and optimal sample size.<\/span><\/p>\n

<\/div>\n

It can also ensure your tools are working as expected, though to save time, we\u2019d recommend simply properly testing your page and tools during development. This ensures you don\u2019t spend additional time on a test that doesn\u2019t help you improve conversions.<\/span><\/p>\n

<\/div>\n

Real world A\/B test examples<\/b><\/h2>\n

The following examples from multinational companies can give you an idea of how the A\/B testing process works.<\/span><\/p>\n

<\/div>\n

Mercedes<\/span><\/a><\/h3>\n

Conventional wisdom says that mentioning price in a luxury ad is tacky. Mercedes used A\/B testing to obtain a definitive, unbiased response. The company created three versions of a luxury ad campaign:\u00a0<\/span><\/p>\n