🔍 Read the full analysis: What The New Opt-Out Means For AI Training And Your Data on ThorstenMeyerAI.com
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced support for a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The signal is voluntary, implementation dates vary, and it does not block crawling, remove content from existing models or prevent use in AI-generated answers.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will support a shared robots.txt signal for website owners who do not want their content used to train generative AI models. The voluntary rule is intended to replace the need for separate training preferences for each participating company, but it does not block sites from being crawled or stop content from informing AI answers.
The proposed instruction uses the user-agent name __usage_ai-training followed by Disallow: /. In practical terms, a publisher adding the rule would be asking participating companies not to use the site’s content for model training. The signal appears in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” written by Anthropic standards lead Kevin Kelley and published October 30.
The companies have not set one shared launch date. Anthropic says it has already begun implementing support, while OpenAI says it aims to adopt the signal “in the coming months.” Google’s documentation points to early 2027. Microsoft says it will honor the training preference from February 1, 2027, as it crawls websites. The companies’ existing crawler-specific controls continue to apply while the shared option is rolled out.
Microsoft also announced a distinct page-level preference for AI Grounding, covering web content used to inform answers in Bing, Copilot and Microsoft 365. It says that preference will take effect on February 1, 2027. Microsoft’s existing NoArchive and NoCache tags will stop counting as training or grounding preferences after March 1, 2027, according to its announcement.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Signal, Not a Block
A common rule could make it easier for publishers to communicate a training preference without maintaining a separate instruction for each AI company. That matters particularly for websites that currently manage controls such as GPTBot, Google-Extended and ClaudeBot, alongside Microsoft-specific tags. A shared signal could reduce missed settings as participating services change their crawlers or policies.
But this is a request, not a technical lock. Robots.txt instructions depend on companies choosing to respect them; they do not prevent other parties from copying or accessing material. The rule also addresses model training, not every use of web content. Microsoft’s separate grounding preference underlines that a site may need to make distinct choices about training and content used in live AI answers.
The distinction matters to both publishers and readers. A site can ask participating companies not to train on its pages while remaining accessible to search engines or potentially usable for answer generation, depending on the company’s separate controls. Microsoft says its preferences do not change eligibility for Bing search results, and Google says Google-Extended does not affect Google Search.
Products Worth Considering

Jackery Solar Generator 1000 v2 with 200W Solar Panel, 1024Wh LiFePO4 Battery

Bose QuietComfort Wireless Bluetooth Noise-Cancelling Headphones, Ice Blue

Amazon Kindle Scribe (16GB) with Premium Pen, Tungsten
As an affiliate, we earn on qualifying purchases.
From Separate Rules to One Draft
Until this announcement, publishers generally had to use company-specific crawler instructions or other controls to express their preferences. The new proposal comes through the AI Preferences working group at the Internet Engineering Task Force, which develops internet standards. The shared rule is described in an individual Internet-Draft, not an adopted standard.
The draft’s status is a key limitation. Its publication does not mean the IETF has approved it or that every AI provider must follow it. At a working group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.” The announcement is therefore a coordination effort among named companies, rather than a binding internet-wide requirement.
Company statements also differ in scope and commitment. OpenAI describes its plan as a “voluntary effort”; Anthropic says implementation has started but will take time across its services. Meta communications chief Andy Stone expressed support for publishers’ ability to state preferences, but did not commit Meta to honoring the signal. The source report said it found no statement from Apple, Amazon, xAI or Perplexity.
““has already started implementing support for this draft””
— Anthropic
Coverage and Enforcement Remain Open
It is not yet clear how many AI providers beyond the four named companies will honor the signal, or how consistently those companies will apply it across their products and services. Meta has voiced support but has not made an implementation commitment in the statements cited. The draft itself has no formal standing in the IETF standards process, and no mechanism described in the source guarantees compliance.
The signal’s practical effect also depends on the exact use of a site’s content. The proposed token concerns training; it does not itself specify an opt-out from live retrieval or AI answers. Nor does it reverse prior training: content already used in a model is not removed by adding the rule. The companies have not provided a shared account of how they will verify compliance or respond to disputes.
Publishers should also distinguish website instructions from controls for personal information or conversations in an AI assistant. This is a site-level preference, not an account setting for a user’s chats. The source material does not establish that the rule governs all data collection or every use of content by participating companies.
Watch the 2027 Rollout
The next test is whether each company implements the signal on the timetable it announced and documents how it applies the instruction. Google’s current target is early 2027; Microsoft’s stated start date is February 1, 2027. OpenAI has not given a specific calendar date, and Anthropic says support is in progress. The IETF working group’s consideration of the draft may also clarify whether it advances toward a formal standard.
For now, publishers considering the setting can add the shared training rule while retaining their existing company-specific controls until providers have implemented the new signal. Microsoft’s March 1, 2027 deadline for NoArchive and NoCache preferences warrants separate attention for sites that rely on those tags. Any deployment should reflect that training, AI answers and search visibility are separate choices, and the shared rule does not guarantee payment or traffic.
Key Questions
What does the shared AI training opt-out do?
It is a robots.txt instruction asking participating companies not to use a website’s content to train generative AI models. The proposed rule uses __usage_ai-training with “Disallow: /”.
Does the rule stop AI companies from crawling a website?
No. The signal expresses a training preference; it does not block crawling or guarantee that a company will comply. Existing crawler-specific rules remain relevant during the rollout.
Does opting out remove a site’s content from existing AI models?
No. Adding the rule does not remove content that may already have been used in training. The announced signal concerns future use for training, not model deletion.
Does the signal prevent a site from appearing in AI answers or search?
Not by itself. The proposal covers training, while use in AI answers can require a separate preference. Google says Google-Extended does not affect Google Search, and Microsoft says its preferences do not change Bing search eligibility.
When will the companies honor the opt-out?
Timing varies: Anthropic says implementation has started, OpenAI aims for the coming months, Google targets early 2027, and Microsoft says February 1, 2027. The signal remains voluntary and is not yet an approved IETF standard.
Source: ThorstenMeyerAI.com
