AI Crawler Access
AI crawler access is whether a website's robots.txt directives and rendering pipeline actually let known AI crawlers read the pages they need in order to describe the business in an answer. If those crawlers are silently blocked, it creates a hard ceiling on visibility in AI-generated answers regardless of content quality.
Many sites unintentionally block AI crawlers through overly broad robots.txt rules, a client-side rendering pipeline that never delivers meaningful content to a non-JavaScript-executing crawler, or CDN/bot-protection rules that treat AI user agents as unwanted traffic. Revvye checks whether a site's robots.txt and rendering pipeline would actually let well-known AI crawlers in if they tried to read the pages that matter — service pages, provider pages, and location pages, not just the homepage.
This check is deliberately narrow and honest about its limits. Revvye does not and cannot guarantee that fixing AI crawler access will get a business cited in ChatGPT, Perplexity, or Google AI Overviews — inclusion in any specific answer experience depends on the engine, the prompt, and the user's context, none of which Revvye controls. What the check does measure is the public-surface reason a business would fail to be citable even if everything else about its content and structured data were perfect: being invisible to the crawler in the first place.
A common real failure pattern Revvye has documented is a robots.txt that blocks exactly the crawler an operator most wants to be cited by, or an advisor/positioning page that's blocked from crawling while only the thin homepage remains visible — meaning the answer engine only ever sees the shallowest version of the business's actual positioning.
See if this is leaking revenue on your site.
The free scan checks this and seven other dimensions in minutes.