What is the Content Signals Policy?
The Content Signals Policy is a robots.txt extension letting a site state how its content may be used, separately from whether it may be crawled.
Traditional robots.txt answers one question: may you fetch this. It has no vocabulary for may you train on it, may you quote it in an answer, may you use it in a summary. The Content Signals Policy adds declarations for those uses, and Cloudflare has inserted it into millions of domains' robots.txt by default.
Its force is unresolved. These are declarations of intent rather than technical controls, and compliance is voluntary in exactly the way robots.txt itself always was. Whether they acquire legal weight is an open question, and anyone stating confidently that they do or do not is ahead of the evidence.
Related
- robots.txtrobots.txt is a file at a site's root that tells crawlers which parts of the site they may fetch. It is a request, not an enforcement mechanism.
- AI crawlerAn AI crawler is a bot operated by an AI company to fetch web pages for training, indexing, or answering a live question.
- llms.txtllms.txt is a proposed convention for a markdown file at a site's root that tells an AI assistant what the site is and which pages matter.
Want to know where you actually stand on this? Run a free visibility check or try the free tools.