A/B testing and search
An A/B test is not a problem in itself. Four published rules frame it: do not serve one version to the crawler and another to people, declare the original address as canonical rather than excluding the variations, redirect with 302 rather than 301, and dismantle the test as soon as it has concluded.
The conflict you think you have, and the one you actually have
A team wanting to improve a page soon runs into an objection: does testing not mean showing the engine something other than what visitors see? The documentation takes the subject from the other end and starts by defusing the fear.
Small interface variations, such as the size, colour or placement of a button, or the text of a call to action, often have little or no impact on the page's search result snippet or ranking.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
The real conflict is therefore not between testing and ranking. It is between the technical means used to test, some of which are documented as violations and others as the correct way of doing the same thing.
The one thing explicitly forbidden
Showing one set of URLs to Googlebot and a different set to humans is cloaking, whether done by server logic, by robots.txt or by any other method, and that holds during a test as well.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
The detail about methods matters as much as the prohibition: the documentation names server logic, robots.txt, and “any other method”. Excluding the crawler from the test so as not to disturb it is therefore the mistake of the subject, and it is taken for what it is, whatever the intention.
Googlebot generally does not support cookies: a test driven by cookies therefore only shows it the version accessible to a browser that refuses them.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
That sentence answers the next question before it is asked: a cookie-driven test only ever shows the crawler the default version. It is not a violation, it is a consequence worth knowing — the version it indexes is the one a cookie-less browser sees, whatever split was planned.
During your tests we exclude Googlebot from the setup via robots.txt: it keeps seeing the original version, so your rankings are protected.
The two settings that make the difference
For a test with several URLs, the documentation recommends marking the original address as canonical on every variation rather than excluding them from the index, the latter sometimes having unexpected bad effects.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
The argument given is one of intent, and it is right: testing a home page, you are not asking the engine to forget your home page, you are asking it to understand that the variations are duplicates of it. Canonical says what you mean; exclusion says something else and can, according to the documentation, produce unexpected bad effects.
A test redirect uses 302 rather than 301: it says the redirect lasts only as long as the experiment and that the original address should stay in the index rather than being replaced by the test page.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
Same logic, different instrument. A permanent redirect announces a move; a temporary one announces a provisional arrangement and leaves the original address in the index. Choosing the wrong code declares permanent what you will dismantle in a fortnight.
The test that never ends
A test must be dismantled once concluded, alternate URLs and scripts included; an experiment running for an unnecessarily long time may be interpreted as an attempt to deceive search engines and acted upon, especially if one variant is served to a large share of users.
officialSource : Google Search Central, “Minimize A/B testing impact in Google Search”
This is the sternest passage on the page, and it targets an everyday situation: the experiment left in place because nobody decided to conclude it. The documentation makes it a question of intent, with an observable criterion — a variant served to a large share of visitors looks less like a test than like a version.
A test redirecting part of the traffic to a different address also moves the clicks recorded in the console to that address: the apparent drop on the original page belongs to the setup, not to ranking.
hypothesis
The status is that of a reading. The documentation does not deal with this effect; it follows from the setup described, since a click landing on a variant address is recorded for that address. A drop observed on the original page during a test therefore deserves to be read with the setup in mind, before any other explanation.
Running an interface test with canonical and a temporary redirect
hypothesis
- Effort
- moderate
- Time to effect
- immediate
- Risk
- low
- Durability
- seasonal
The effort lies in configuring the testing tool, not the content. The delay is immediate for what the test measures. The risk deserves two notches, because the wrong means are documented as violations and the mistake is easy to make. Durability is low, and rightly so: a test is provisional by definition, and its value lies in what you do with it afterwards.
What does not work, or no longer works
Excluding the crawler from the test, first. It is named as cloaking, robots.txt included.
Then excluding the variations from the index out of caution. The documentation recommends canonical and notes that exclusion can produce unexpected effects.
Finally, letting a test run because it bothers nobody. The page covers that case explicitly and treats it as a possible attempt to deceive, especially when one variant is served to a large share of visitors.
Sources
- Minimize A/B testing impact in Google Search — Google Search Centralaccessed on
- Google Search spam policies — Google Search Centralaccessed on
Block revalidated on
Frequently asked questions
- Should a test be hidden from the crawler?
- No, and it is the one move the subject explicitly forbids. Showing one set of URLs to the crawler and another to people is cloaking, whatever the method used, robots.txt included. The documentation notes that breaching the policies can get a site demoted or removed from the results.
- Does an A/B test hurt ranking?
- The documentation says the opposite for small interface variations: colour, size or placement of a button, and call-to-action text often have little or no impact on the snippet or the ranking. What causes problems is not the test but badly chosen technical means of running it.
- Why a 302 redirect rather than a 301?
- Because it says the redirect is temporary. The engine then keeps the original address in its index instead of replacing it with the test page. A 301 during an experiment amounts to announcing a permanent move to an address you will dismantle next week.
- Should variations be excluded from the index?
- The documentation recommends canonical rather than exclusion, and gives the reason: testing a home page, you do not want the home page to stop being indexed, you want the variations grouped with the original. It notes that exclusion can sometimes have unexpected bad effects.
- How long may a test run?
- Long enough to conclude, no longer. The documentation asks that alternate URLs and scripts be removed as soon as the test ends, and warns that an experiment dragging on may be read as an attempt to deceive, especially when one variant is served to a large share of users.