What is server-side rendering, and does it matter for AI crawlers?
Server-side rendering means the HTML arrives complete from the server. It matters because most AI crawlers do not execute JavaScript, so content injected in the browser is invisible to them.
Googlebot renders JavaScript, though on a delay. The AI crawlers largely do not. A page whose content is fetched client-side may therefore be fully visible in Google and effectively empty to GPTBot, ClaudeBot and their equivalents.
The check is simple and worth running before any deeper diagnosis: fetch your page without executing scripts and see what text is present. If the substance appears only after hydration, no amount of structured data or robots.txt permission will help, because there is nothing there to read.
Related
- AI crawlerAn AI crawler is a bot operated by an AI company to fetch web pages for training, indexing, or answering a live question.
- Structured dataStructured data is machine-readable markup, usually schema.org JSON-LD, that states explicitly what a page and its subject are.
- IndexationIndexation is whether a search engine has stored your page and is willing to return it. Being crawled is not the same as being indexed, and being indexed is not the same as ranking.
Want to know where you actually stand on this? Run a free visibility check or try the free tools.