What it is

Content Signals adds a line to robots.txt that states how your content may be used: for search (search), as input to an answer an agent gives (ai-input) and to train models (ai-train). Each is yes or no.

An analogy

Labels on a shelf: you may look, you may quote, you may not copy the whole book.

At a glance

Owner
Cloudflare (released CC0)
Maturity
Emerging Published September 2025. On by default in Cloudflare’s managed robots.txt, which gives it wide reach quickly.
Where it lives
contentsignals.org
Framework domain
Discover: Be found
What it serves
  • State how your content may be used
  • Allow answers while declining training

Who should care

The roles this standard touches most. See all the roles on the personas page.

  • Marketing and brand. Own the story agents tell: accurate facts, positioning, reputation and presence where agents look.
  • Legal, risk and security. Set the policies: which agents may do what, identity, data, disclosure and liability.
  • Engineering and product. Build the access, data, endpoints and actions that agents use, on open standards.

Framework personas →

How to adopt it

  1. 1

    Decide your position on search, ai-input and ai-train with legal and marketing.

  2. 2

    Add the signal line to robots.txt.

  3. 3

    Test that the file still parses and that the lines say what you meant.

The Institute’s view

Adopt with care

It is cheap and clear. It is a statement of preference, so it does not stop a crawler that ignores it. It may be superseded by the IETF work on the same problem, so keep your policy decision separate from the syntax.

Standards change fast. Last reviewed November 2026.

← Back to the Standards Map