Do AI answer crawlers skip content inside collapsed accordions?

You updated your product pages or FAQ section to use collapsible accordions. Visually, the page is clean: a user clicks a header, the element expands, and the text appears. Then you run a prompt in ChatGPT asking for a product recommendation in your niche, and your brand is missing. Someone on your team asks whether ChatGPT skipped your content because it was hidden inside a collapsed accordion.

To answer that question, you need to look at how live search retrieval works in generative AI engines. The answer depends entirely on whether your accordions exist in the raw HTML document returned by your web server or whether they require client-side JavaScript to render.

Raw HTML versus rendered DOM in live retrieval

When a user asks a prompt that requires real-time web retrieval, an AI engine dispatches an answer crawler to pull content from the web. OpenAI uses ChatGPT-User and OAI-SearchBot for this purpose. Anthropic uses Claude-SearchBot and Claude-User, while Perplexity uses Perplexity-User and PerplexityBot.

These answer crawlers are completely different from training crawlers like GPTBot, ClaudeBot, or Google-Extended. Training crawlers crawl the web in the background to build datasets for future base models. Blocking GPTBot in your robots.txt file prevents OpenAI from using your content to train future models, but it does not stop ChatGPT-User from reading your site right now during a live web search.

Answer crawlers operate under extreme latency constraints. A user waiting for an answer expects a response in a few seconds. To meet that speed target, answer crawlers do not launch full headless web browsers. They do not initialize JavaScript runtimes, construct a full rendered Document Object Model (DOM), or trigger user interaction events like clicking an accordion toggle button.

Instead, an answer crawler makes a simple HTTP GET request to your URL and reads the raw HTML stream sent back by your server.

If your accordions are structured using standard CSS where the full text payload is loaded into the initial HTML document and toggled visually using CSS properties like display: none or visibility: hidden, the answer crawler receives every single word. Language models process raw text tokens extracted from markup. They do not process visual layout layers, so CSS rules that hide text from human eyes do not hide text from an answer crawler.

If, however, your site relies on client-side JavaScript frameworks to inject text into the document only after a user clicks a button, the raw HTML stream contains empty wrapper tags. In that specific scenario, an answer crawler will miss the hidden content because it never executes the script required to fetch and display the body copy.

How to test what ChatGPT-User actually sees

You do not need to guess how an answer crawler handles your page layout. You can inspect the exact payload your server delivers to ChatGPT-User directly from your terminal using curl.

Run this command against your URL:

curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ChatGPT-User/1.0; +https://openai.com/bot)" -sL https://yourdomain.com/your-page | grep -i "your accordion text"

This command issues a request using the exact user-agent string that OpenAI sends during live search retrieval, follows redirects, and searches the returned string for your copy.

If the command prints your accordion content surrounded by HTML markup, ChatGPT-User receives that text every time it fetches the page. It does not matter if the surrounding tags carry classes like accordion-content-hidden or inline styles like display: none;. The tokens are in the stream, and the engine can parse them.

If the command returns nothing, your text is being injected dynamically by JavaScript, or your server is treating the crawler differently than a regular browser request.

Server-level blocking is a far more common reason for missing citations than accordion design. In our crawler index snapshot from August 8, 2026, across a panel of 54,082 domains, 33,670 returned a readable robots.txt file. Out of those 33,670 readable domains, 5,497 block at least one AI agent, but only 2,576 block at least one answer crawler.

Many site owners accidentally block live retrieval while trying to opt out of training. Per agent, out of 33,670 readable domains:

• GPTBot (training) is blocked by 5,080 domains
• ClaudeBot (training) is blocked by 4,603 domains
• Google-Extended (training) is blocked by 4,275 domains
• PerplexityBot (search) is blocked by 2,147 domains
• ChatGPT-User (search) is blocked by 2,074 domains
• OAI-SearchBot (search) is blocked by 1,579 domains
• Perplexity-User (search) is blocked by 1,402 domains
• Claude-SearchBot (search) is blocked by 1,400 domains
• Claude-User (search) is blocked by 1,384 domains
• Googlebot (other) is blocked by 610 domains

In a census of active Y Combinator companies conducted on August 7, 2026, 3,755 out of 4,226 active company domains had readable robots.txt files, and 253 blocked at least one AI crawler. If your infrastructure blocks ChatGPT-User or Perplexity-User, the engine receives a 403 or 401 status code, and no amount of DOM restructuring will change the result.

Why expanding accordions will not fix your visibility

There is a minor technical concession worth making: uncollapsed, plain HTML paragraphs are slightly cleaner to parse than deeply nested accordion markup. Stripping away unnecessary wrapper tags and complex DOM trees reduces noisy markup tokens in the payload fed into the model context window.

However, moving all your accordion content into open body text will not guarantee higher recommendation frequency or instant citations in AI answers.

Engine behavior is non-deterministic. Generative models sample from probability distributions. An engine might pull your URL into its context window during search retrieval, evaluate the text, and still omit your brand from the final output because another source was weighted higher for that specific prompt run.

Do not buy agency packages or software plugins that promise to fix your AI search visibility by refactoring your HTML components, adding custom schema markup, or generating llms.txt files. We have seen no evidence that schema markup or llms.txt files increase citation frequency in live AI answers, and any vendor claiming to guarantee a specific citation or ranking spot in ChatGPT is selling snake oil.

If you want to know whether your brand appears in AI engines, you must measure output across multiple runs rather than relying on a single query or a raw HTML audit.

At Standing, we track recommendations across five major engines: ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews. We use a fixed prompt basket run five times per question per engine, calculating a Wilson score interval for every result. A raw score like 34 means nothing on its own; a proper measurement must express variance, such as 34, plus or minus 6, within a 95% confidence interval. Competitors run each prompt a single time and report a static number, ignoring the fact that running the prompt again ten seconds later can yield a different answer.

Our plans start at Track for $100 per month across 3 domains, Optimize at $300 per month for 5 domains with custom prompt creation, and Agency at $500 per month for 50 domains with a monthly re-scan.

If your accordions load their text into the initial HTML document, answer crawlers can read them. Keep your site layout accessible to human users, test your server responses with curl, and evaluate your visibility based on statistical output distributions across multiple runs rather than tweaking visual CSS components.