# Discover domain

> Agents can reach you wherever they look: your site, directories, feeds, marketplaces and the assistants people already use.

Source: https://agentrelations.org/framework/domains/discover

## The outcome and why it matters

Agents decide what to read before they decide what to say. If they cannot reach your pages, or never learn you exist, you are left out of the answer, and nobody notices because the person never saw the option. Discovery covers the access you allow, the maps you publish and the places agents already look.

An analogy

Think of a shop. Discovery is the street address, the sign on the door and the listing in the phone book. A shop with the doors locked and no listing has no customers, however good its products are.

Led by: Marketing and brand. See the [responsibility matrix](https://agentrelations.org/framework/personas).

## Capabilities

5 things an organization does in this domain. Standards named here are explained in the Standards Map.

Capability 1

### Crawler and agent access policy

A deliberate decision about which agents may read what. Agent traffic comes in several kinds: crawlers that collect content for training, crawlers that build search indexes and fetchers that load a page because a person asked an assistant to.

What good looks like

Access is set per kind of agent and per area of the site, written down and reviewed each quarter. Nothing is blocked by accident, for example by a firewall rule or a bot-protection default.

Typical standards and techniques

- robots.txt (RFC 9309) with rules for AI user agents such as GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and CCBot
- Control tokens such as Google-Extended and Applebot-Extended
- IETF AI Preferences (AIPREF) work on usage signals

Capability 2

### Site maps and freshness signals

A complete, current list of the pages and files you want agents to find, and a signal when they change.

What good looks like

The sitemap lists every canonical public page with accurate last-modified dates. Pages that agents need load without scripts and without a login.

Typical standards and techniques

- XML sitemaps (sitemaps.org)
- HTTP caching headers and Last-Modified
- IndexNow for change notification

Capability 3

### An agent-readable guide to the site

A short file that tells an agent what the site contains and which pages matter most, plus clean text versions of key pages.

What good looks like

A maintained guide points to the pages and documents a person would send a newcomer. Key pages are also available as simple Markdown or plain text.

Typical standards and techniques

- llms.txt (a community proposal, a Markdown file at /llms.txt)
- Markdown or plain-text versions of pages
- Canonical URLs (rel=canonical)

Capability 4

### Presence where agents look

Agents also learn about you from places you do not run: business listings, review sites, marketplaces, registries, encyclopedias and other organizations’ pages.

What good looks like

A list of the sources agents rely on for your sector, with owners, and the facts in them match your own. Claims are corrected where they are wrong.

Typical standards and techniques

- Business profile and listing platforms for your sector
- schema.org sameAs links between your profiles
- Public registries and knowledge bases (for example Wikidata)

Capability 5

### Directory and registry listing

Being listed in the places built for agents to find services: tool registries, agent directories and marketplaces.

What good looks like

Your agent-facing services are listed in the registries that matter to your audience, with accurate descriptions and a contact.

Typical standards and techniques

- Model Context Protocol (MCP) server registries
- Agent2Agent (A2A) Agent Cards published at a well-known address
- Industry directories

See the [Standards Map](https://agentrelations.org/standards) for who owns each standard and how mature it is. To learn the skills, see the [Specialist certification](https://agentrelations.org/learn/certification/specialist) (Marketing and Engineering).

## KPIs

Defined precisely, so two organizations count the same way.

Agent reach

The share of your canonical public pages that identified agents fetched successfully (HTTP 200) in the period, out of all pages in your sitemap.

Unintended blocks

The number of agent user agents you intend to allow that received a block (403, 429 or a challenge page) in the period. Target: zero.

Source coverage

The share of the sources agents rely on for your sector, from your documented list, where a correct, current profile of you exists.

## What each maturity level looks like here

1. 1
Level 0: Closed
Agents are blocked, often by accident, or cannot load your pages at all.
1. 2
Level 1: Visible
Public pages are reachable. Access rules exist but nobody has reviewed them for agents.
1. 3
Level 2: Described
A reviewed access policy and a current sitemap. Key sources outside your site are listed and correct.
1. 4
Level 3: Conversational
An agent guide such as llms.txt and plain-text versions of key pages. Your services are listed in agent registries.
1. 5
Level 4: Callable
Discovery runs as a routine: access reviewed quarterly, listings checked monthly, unintended blocks alerted within a day.
1. 6
Level 5: Trusted
Agents find you first in your category, and you adjust access policy from evidence about which agents bring outcomes.

[See all domains side by side](https://agentrelations.org/framework/maturity) or take the [self-assessment](https://agentrelations.org/framework/maturity#self-assessment).
