Our digital marketing blog offers you a vantage point into the world of online marketing and its many themes, like intercultural competence, B2B marketing, SEA, SEO, social media marketing, web analytics and more.


SEO Experiments: With This A/B Testing You Can Systematically Optimise Your Website and Avoid Any Loss of Ranking





Search engine optimisation plays an important role in attracting visitors to a particular website. If one doesn’t want to squander growth potential, it is wise to include SEO experiments as a regular feature of the online marketing strategy.

SEO experiments do not function in the same way as “traditional” A/B tests for conversion pages or emails; they require a different framework



What is the difference between SEO experiments and classic A/B tests?


For a conversion rate split test (or A/B test), visitors are randomly sent to either version A or version B of a web page where different call to actions can be tried out and various product images and other features tested.

To test a conversion page you need three things:
•    users that are randomly divided into two test groups,
•    2 versions of the webpage
•    and a parameter (in this case, leads) in order to ascertain which page is “best” .


Split Test
This is how you can determin which page is the most effective


However, this method cannot be directly applied for a SEO split test. If you were to place several different versions of a web page online in order to ascertain which gets more traffic, Google would immediately set off an alarm. The pages would end up destroying each other and eventually Google would not index one of them, so that the split test would not produce any worthwhile result.

There is, however, a framework that you can use to test whether and how modifications can affect the ranking of a page.


What comprises a SEO split test?


At the beginning of this year Pinterest laid the foundations for this test when it introduced its Framework für SEO-Experiment

The basic idea is that instead of producing two versions of one page, one creates several pages that are very similar.


There are three steps to a SEO experiment:
•    Identify a group of pages that need to be improved. Several, similar pages are necessary to carry out the SEO experiment.
•    Modifications should be made to the randomly (!) selected pages of this group. Important: a control group must remain unchanged.
•    Decide on the parameters for your evaluation of the test results. Some possible options would be: ranking, number of visitors, average duration of visit, leads etc.
We carried out such an experiment on a small group of similar pages. The following modifications were tested:
•    Group 1: considerably more content in the form of text
•    Group 2: less text, more content in the form of optimised videos and images
•    Group 3: unchanged control group

The fundamental question was: which changes attract more visitors to the page and how long does a SEO experiment take when there are comparatively few page views (max. 1000/month)?


The results of the perfomance test


From our SEO experiment we concluded that if there are a lot of visitors to the site, the results of the SEO split test can already be assessed after a few days (see Pinterest); however, if the numbers are low, then results are only seen after 1-2 weeks.

If there are major changes, the evaluation is usually fairly simple. The diagram below shows changes to the number of visitors in relation to the control group. The figures for our test website were pretty low to start with, which explains the high percentages.


AB Test


Both alterations improved the performnace but the pages with


siginficantly more text outperform the other groups.



A small experiment with large benefits


If you wish to make any alterations to your website that have the potential to effect the search engine rankings, you should test the changes first on a subgroup of pages. The Pinterest experiments demonstrated that a technical modification, for example, resulted in a considerable loss of traffic.

If a gradual loss of traffic is observed after changes have been made across all pages of a site, it cannot be definitively concluded that this was caused by the changes. Traffic fluctuates all the time and the Google algorithms also change regularly. However, if the test is carried out with just a few pages, useful conclusions can be drawn from the performance results.


There are two other important advantages of SEO experiments

  1. Marketing accountability: the creation of new content for 10 pages costs 10 times as much as for one page. If only a few pages are modified, the investment in time and money is manageable. In addition, the resulting data is reliable and, if a positive effect is recorded, this can be used to justify further investment. 
  2. Marketing attribution: if all the site pages are altered at the same time, it is not possible to clearly identify the effect of the individual pages. If  a SEO experiment is first carried out, it is possible to precisely ascertain the effect on the target parameters and to calculate how many additional conversions can be attributed to each single modification (and thereby, also to the investment made).   

In conclusion, large scale optimisation will set off a process where cause and effect cannot be clearly identified. It is much better to test innovations on a small number of pages first, in order to have a clear picture of the effect of the changes on the SEO ranking.