Let AI assistants read your site
Can AI even read your site? Your robots.txt says yes. Your CDN may disagree. We find the layer turning the engines away, clear it, and re-test all ten agents against your live server.
Not sure this is the right service?
- GPTBot403 · blocked at CDN
- PerplexityBot403 · WAF rule
- ClaudeBot200 · reading
- Google-Extended200 · reading
- OAI-SearchBot200 · reading
- Delivered in 3 working days
- Full refund if late
- Re-test included
- Response codes in writing
What you get
One fixed price. The fix is the quick half; the verification against your live server is what you're paying for.
- 10 AI agents tested against robots.txt, CDN and WAF
- The exact rule behind every block, quoted
- A corrected robots.txt, written and applied for you
- All 10 re-fetched after deploy, response codes in writing
What this covers
A free checker reads your robots.txt. We audit the layers underneath it, where most real blocks live.
The blocking rule quoted
Not "you're blocked": the exact CDN or WAF rule doing it.
Fixed and applied
We apply it, or send the exact diff if you'd rather deploy yourself.
Training kept separate
Allow the agents that cite you without agreeing to the ones that train.
Where this ends
We clear every block between the engines and your server. What they do with the pages they can finally read is Earn work.
How it works
Order to verified access in three working days. The report shows what your live server actually returned, not what the config predicted.
- Day 0You order
Five-minute intake: your domain, your CDN, and the access level you're comfortable giving.
- Day 1The audit
All ten agents tested at all three layers. Every block found, and the rule behind it quoted.
- Day 2The fix ships
Corrected robots.txt and the CDN or WAF change, applied by us or sent as an exact diff.
- Day 3 · 29th SepThe re-test
All ten agents re-requested against your live server. The job isn't finished until each one gets through.
What arrives
A report you can hold us to: the block, the fix, and the proof it worked.
The rule, not a guess
Free checkers read the file and stop. This names the CDN or WAF rule actually refusing each agent.
The before and after
Every agent's response code from your live server, on both sides of the fix. 403 → 200 or we're not done.
Cite and train, separated
Which agents affect whether you're cited and which only feed training, so you allow the first without the second.
What we stand behind
- Delivered or refunded
- The date is on your order. Late means the refund is automatic, not requested.
- Every agent gets through
- The re-test after your deploy is included; unfinished until each agent clears.
- Checkable, in writing
- Every code is the one your live server returned, so you can re-run any of them.
Questions
Because the fix is the quick half and the verification is not. Correcting robots.txt, a CDN rule or a WAF rule is an afternoon; the corrected file then has to be live on your own server before the ten agents can be re-fetched against it, and the response codes in the report are the ones your live server actually returned rather than the ones the configuration predicted. A rule that reads correct in a dashboard and a rule the server applies are different things, and only the second one is worth paying for.
Usually not robots.txt at all. Bot protection at the CDN, a WAF rule, or Cloudflare's bot-fight mode refuses the answer crawlers regardless of what your file permits, and none of that appears in the file a checker can read, which is why a free scanner can tell you that you are blocked but not by what. We audit the layer underneath, apply the change, and re-test all ten agents against your live server.
Run the free crawler checker: it names the agent and the layer. Of the 33,670 domains on our index with a readable robots.txt, 83.7% disallow none of these agents (a further 20,412 could not be read and are excluded from both sides), which is how we recognise a real block when one shows up. If the checker flagged an agent on your domain, you are in the 16.3% this is built for.
We re-test every agent against your live server after you deploy, and the job is not finished until each one gets through. That re-test is included, not an extra.
No, and the distinction is the work. Some agents affect whether an engine can cite you in an answer, others only affect training. We separate them so you can allow the first without agreeing to the second.
When the free checker comes back clean, and when it names one line in a robots.txt file you control and changing that line yourself works. This is priced for the case where the file already looks right and the agents are still being refused, which is the case a file-reading checker cannot diagnose.
Each of the ten agents re-requested after deploy, with the response shown.
The same questions before and after, each with its own confidence band. A move counts once it clears your domain's own drift, which is why the band is published rather than just the score.
A corrected robots.txt, applied and re-tested. We clear every block between the answer engines and your server, at the CDN and WAF layer as well as in robots.txt, and re-test all ten agents against your live server afterwards. What an engine then does with the pages it can finally read is the job of the Earn services.
On the record
“We've been focused on SEO for years, so I was initially skeptical about adding another tool. But seeing how differently AI platforms talk about our company compared to Google search was pretty eye-opening. Standing makes that difference much easier to see. I particularly like being able to track competitors and see what sources are being cited.”
Let the engines back in.
$300 one-time, delivered by 29th Sep. Every agent re-tested against your live server, every response code in writing.