What is a denominator?
A denominator is the total group a rate gets measured against, and changing what counts as that total changes the percentage without changing anything real.
Any rate, a citation rate, a block rate, a share of voice, is a fraction, and the denominator is what's underneath the line, the full group being counted. A block rate of 16% only means something once you know what the 16% is out of: all domains contacted, all domains that returned a readable response, all domains in some registry. Swap the denominator and the same underlying data produces a different headline number.
This is where a lot of quoted statistics quietly go wrong. If a crawl includes sites that timed out, returned an error, or blocked the crawler itself, and those failures get dropped from the denominator rather than counted as "status unknown," the resulting percentage describes only the domains that were reachable, not the full population anyone assumed it covered. Excluding the unreadable domains isn't dishonest by itself, but reporting the rate without saying what was excluded is.
The honest version of any rate states its denominator plainly: not "16% of domains block a crawler" but "16% of the 33,670 domains with a readable robots.txt block a crawler," with the tens of thousands of unreadable domains named as excluded rather than folded silently into either side.
Related
- Selection biasSelection bias is when the group you managed to measure differs systematically from the group you meant to draw conclusions about, because of how members ended up included or excluded.
- robots.txtrobots.txt is a file at a site's root that tells crawlers which parts of the site they may fetch. It is a request, not an enforcement mechanism.
- AI crawlerAn AI crawler is a bot operated by an AI company to fetch web pages for training, indexing, or answering a live question.
- Sample sizeSample size is the number of prompts, queries, or pages included in a single measurement, and it sets how much random noise a result carries.
Want to know where you actually stand on this? Run a free visibility check or try the free tools.