Skip to content

Research

We run a frozen panel of 5,000 domains and re-check the same ones on a schedule. Every figure is published with the population it came from, the domains we had to exclude, and what it does not show. A number without its denominator is not a finding, and this category is full of them.

5,000
Domains in the panel
10
Crawlers checked on each
Never
Membership changes

Published

August 2026

How many sites block AI crawlers

18.2% of 1597 domains disallow at least one AI crawler. Published estimates elsewhere range from 3.5% to 88.9%, because almost none of them travel with the population they were measured on.

Read it

The panel behind these is frozen and re-run on the same domains, so each edition reports a change rather than a fresh sample. Nobody can buy a measurement taken last month, which is the only reason a company this size can own a dataset at all.