Skip to main navigation Skip to main content Skip to page footer
Deutsch English Français 中文
Home / Blog / Blog Details

Blocking AI training without disappearing from AI answers

OpenAI, Google, Microsoft and Anthropic will support a shared opt-out from AI training. What this means for visibility in AI answers and what B2B companies should look out for now.

robots.txt with two switches: AI training is switched off, AI answers is switched on, both linked by a chain and a warning symbol; steffi winks and points at the switches, effie stands next to them

The shared AI opt-out and its impact on visibility

On 30 October 2026, OpenAI, Google, Microsoft and Anthropic announced that they will support a shared signal in robots.txt. It is intended to let website owners exclude their content from AI training by all four providers in a single step, rather than having to block a separate crawler for each provider. The basis is a draft at the IETF, and implementation will continue into 2027.

For greater control over your own content, this is a step forward. For visibility in AI answers, however, it holds a trap worth knowing about.

Training and grounding: two different things

In training, content flows into future models and shapes what a model "knows" about a brand, a product or a topic. In grounding, an AI system retrieves a page live to answer a current question and cites it as a source. This is exactly the kind of visibility that matters in AI search.

An opt-out from training therefore affects future models. Whether a page is cited in an AI answer today, by contrast, depends on grounding. What matters is whether the two can be controlled separately.

What the providers are implementing

Microsoft separates the two: training and grounding can be controlled independently of each other, including for individual pages. Until now, Bing has also treated the NoArchive and NoCache directives as AI preferences. From 1 March 2027 this will no longer be the case, so anyone using them for this purpose will need to switch.

Google links the two for Gemini: training and use in the Gemini app currently depend on the same setting. Blocking training can therefore also remove a site from answers in the Gemini app. AI Overviews and AI Mode have their own setting in Google Search Console. Support for the shared signal has been announced for early 2027.

OpenAI and Anthropic also intend to respect the shared training signal.

What B2B companies should do now

For industrial companies that want to be found in AI answers, the rule is: decide first, then block.

  • Take stock: Which AI crawlers does your robots.txt currently allow or block, such as GPTBot, Google-Extended, ClaudeBot or PerplexityBot? Many blocks were set across the board at some point and never reviewed.
  • Check meta directives: Does your website use NoArchive or NoCache? If so, their effect on Bing will change on 1 March 2027.
  • Make a conscious decision: For most B2B companies that want to benefit from AI recommendations, the value of visibility outweighs the rest.
  • Measure the impact: A citation analysis shows whether your content is cited in AI answers, before and after every change.

How to prepare your website technically for AI systems, for example with an llms.txt, is covered in our article Making TYPO3 agent-ready.

Not sure what your website is currently signalling to AI crawlers?

➡️ More on AEO & GEO
➡️ Citation analysis with aiva

Questions about robots.txt and AI crawlers?
➡️ Get in touch

As of November 2026.
Sources: Microsoft Bing Blog, Google Search Central Blog, OpenAI, Search Engine Journal

Back