medium.com blocks AI training crawlers, but can still be cited
medium.com disallows 2 AI crawlers in robots.txt, all of them training crawlers. That keeps its content out of the models without stopping an assistant citing it. Checked 2026-08-07.
Disallow: /
This does not affect citation
Every disallowed agent here is a training crawler. medium.com has opted out of the models while remaining quotable, which is a deliberate and common position. Any tool reporting this as "invisible to AI" is wrong.
Training crawlers
Blocking these keeps content out of the models. It does not stop an assistant citing the site.
- GPTBotChatGPT
trains and grounds OpenAI models
- ClaudeBotClaude
trains and grounds Anthropic models
Permitted
OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot, PerplexityBot, Perplexity-User, Google-Extended, Googlebot
What this does and does not show
Method. One request to medium.com/robots.txt on 2026-08-07, parsed for Disallow rules applying to each of the 10 named agents. No JavaScript, no login, nothing that cannot be reproduced with curl in a few seconds.
This is a snapshot. robots.txt changes whenever the site owner edits it, and this page reports one day. Anyone can re-run the same check live, and if it disagrees with what is here, the live result is the true one.
Access is not citation. A permitted crawler may never fetch the page, and a fetched page may never be quoted. Equally, a blocked crawler does not make a site invisible: content an engine learned before a rule was added does not disappear, and other sites writing about medium.com remain readable.
Bot protection. 7 live probes were refused by a CDN rather than by robots.txt. We send a crawler's name from our own address, which is exactly what an impersonator looks like, so that is the protection working rather than proof the real crawler is shut out.
Check your own site
The same check runs on any domain, free and without an account. It reports training and search crawlers separately, because collapsing them is the mistake this page exists to correct.