AI crawler access describes what a site's public robots.txt, response headers, and page directives declare for known AI-related user agents. The declaration can permit or restrict access, but it does not prove that an agent visited, indexed, or cited the page.
AI Crawler Access
The operational definition, the public evidence boundary, and the nearby concepts an operator needs before using the term in a diagnosis.
Definition before diagnosis.
What the term can—and cannot—support.
Read the definition as a contract. It keeps a useful public observation from expanding into an outcome the scan cannot see.
- 01
Revvye's public checker requests the submitted URL and related public directives, then evaluates the declared rules against its source-controlled crawler catalog. The result is bound to the tested URL, response, time, and known-agent list.
- 02
CDN behavior, authentication, rate limits, regional responses, later crawler choices, and multi-page coverage may differ from that observation. A permitted directive is therefore a prerequisite signal, not a visibility guarantee.
The term travels with its limit.
- 01Observe
Inspect the submitted public surface and the declarations it returns.
- 02Preserve
Keep the rule, page signal, severity, and stated limitation together.
- 03Exclude
Do not infer private revenue, conversion, delivery, or customer behavior.
Inspect the public path. Keep the private result private.
The free scan begins with the page you submit and routes its result through a private email-linked intake. It does not access private analytics or CRM data.
Public pages only. The $97 report comes after the result.