What is Google-Extended?
Google-Extended is a robots.txt token that controls whether your content trains Gemini and grounds its answers. It is not a crawler and blocking it does not affect Google Search ranking.
The confusion here is structural. Google-Extended is not a user-agent that fetches anything; Googlebot does the crawling, and Google-Extended is a separate control over what those crawled pages may be used for. So disallowing it removes you from Gemini training and grounding while leaving your search rankings untouched.
That design is deliberate and it means the usual fear does not apply: you cannot accidentally deindex yourself from Google Search by blocking Google-Extended. Our index finds it disallowed on 4,275 of 33,670 readable domains, making it the third most-blocked token after GPTBot and ClaudeBot.
Related
- AI crawlerAn AI crawler is a bot operated by an AI company to fetch web pages for training, indexing, or answering a live question.
- robots.txtrobots.txt is a file at a site's root that tells crawlers which parts of the site they may fetch. It is a request, not an enforcement mechanism.
- AI OverviewsAI Overviews are Google's AI-generated summaries shown above the traditional results for some searches.
Want to know where you actually stand on this? Run a free visibility check or try the free tools.