OpenAI, Google, and Microsoft commit to shared robots.txt standard for AI training opt-outs
What happened.OpenAI, Google, and Microsoft announced support for a unified policy token developed through the IETF to manage generative AI training preferences across participating platforms in a single directive.
Why it matters.Maintaining dozens of fragmented bot exclusion directives created severe operational friction for digital publishers protecting proprietary data. Standardizing training preferences at the network boundary allows engineering teams to control model scraping while maintaining organic discoverability across commercial search interfaces.
