Google, Microsoft and OpenAI back a one-line robots.txt opt-out for AI training

Anthropic's Kevin Kelley has published an Internet-Draft for a one-line robots.txt signal that tells AI companies a site does not want its content used for model training. Google, Microsoft and OpenAI have said they will support it. It is a draft, not a standard, and it states a preference rather than blocking access.

What was proposed

The draft is titled "Machine-Readable Training Preference Signal" (draft-kelley-aipref-training-preference-signal-00), dated 30 October 2026. It is an individual submission: the IETF has not endorsed it and it has no formal standing in the standards process.

A site adds a group to its robots.txt for the policy token `__usage_ai-training`. The draft's example expresses a preference against training for every path except a press kit:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

Who says they will support it

According to The Next Web, the companies' positions are:

• Google plans to support the signal in 2027. If a site uses the token without mentioning Google-Extended, Google will treat it as a block on Google-Extended, its own AI training control.
• Microsoft will honour the preferences from 1 February 2027, replacing the NoArchive and NoCache tags it has used since 2023. It is also adding a separate page-level setting for grounding, and warns that blocking grounding will significantly reduce a page's visibility in AI-generated experiences, without affecting Bing search rankings. It says the preferences do not establish licensing or payment arrangements.
• OpenAI aims to implement the signal in the coming months. Existing controls such as GPTBot still apply.
• Anthropic proposed the draft and says it will complement its existing crawler controls, with no timeline given.

What it does not do

The signal is a statement of preference, not a crawling order. A site with no such group has expressed no preference, and rules written for all crawlers do not count toward it. A private agreement between a site and an AI company can override it, and it creates no licence or payment. Because it is a draft, it may change.

What it means for businesses

For most business websites, nothing needs to change today. Our own robots.txt, for example, allows all crawlers and expresses no AI training preference, because we want our pages to be found and cited. That is a choice, not a default to copy.

The questions worth answering before 2027:

• Which content is public marketing, and which is licensed or paid? Marketing pages usually gain from being cited by AI tools. Paid reports, members-only material and licensed images are candidates for an opt-out.
• Who edits your robots.txt? Many CMS and CDN tools generate it automatically, so check what yours says before adding anything.
• Do you rely on it for privacy? Do not. robots.txt is a public request, not a lock. Private data needs sign-in and permissions.

If you want help deciding what to expose and what to protect, our SEO service covers crawler rules and AI visibility, and our website management team can set up the file. You can also ask us directly.