Can AI actually read your site?
A blocked crawler cannot cite you, no matter how good your content is. Check all 10 agents in about three seconds.
Free, no signup. Reads robots.txt and requests your homepage as each crawler.
What this checks
- GPTBottrains and grounds OpenAI models
- OAI-SearchBotbuilds ChatGPT's search index
- ChatGPT-Userfetches a page when a user's question needs it
- ClaudeBottrains and grounds Anthropic models
- Claude-Userfetches a page during a Claude conversation
- Claude-SearchBotbuilds Claude's search index
- PerplexityBotbuilds Perplexity's index
- Perplexity-Userfetches a page to answer a live question
- Google-Extendedcontrols Gemini grounding and training
- Googlebotfeeds Google's index, including AI Overviews
Common questions
- What is GPTBot?
- GPTBot is OpenAI's crawler. It reads pages to train and ground OpenAI's models, including ChatGPT. Blocking it in robots.txt removes your site from that pool.
- Does blocking AI crawlers affect my Google ranking?
- Blocking GPTBot, ClaudeBot or PerplexityBot has no effect on Google Search, because those are different crawlers. Google-Extended is the exception worth understanding: it controls Gemini grounding and training but does not affect Google Search ranking either. Googlebot itself is the one that does both, so disallowing it removes you from Search and from AI Overviews together.
- Why does a crawler show as blocked when my robots.txt allows it?
- robots.txt is a request, not a wall. A CDN or firewall rule that rejects anything without a browser fingerprint will return 403 to a crawler regardless of what robots.txt says. That is why this tool also requests your homepage as each agent instead of only reading the file.
- If every crawler is allowed, will AI recommend me?
- No. Access is necessary, not sufficient. It means a crawler is permitted to read you; whether an engine then names you in an answer depends on what it finds and what else it has read.
Access is the floor, not the goal
This tool answers whether an engine is allowed to read you. It does not answer whether one recommends you, which is a different measurement and the one we built Standing for.
Run a free visibility check