Do cross-origin iframe embeds get read by AI search engines?

You embedded your pricing table, interactive comparison calculator, or customer review widget inside an iframe hosted on an external domain. In a standard browser, the component renders seamlessly. When a user opens ChatGPT or Perplexity and asks for a direct comparison of your pricing or features, the model speaks as if that embedded data does not exist.

The problem is not that the model is choosing to ignore your product details. The problem is that the crawler fetching web context for the model never receives the text inside the frame.

How answer crawlers handle cross-origin iframes

Answer crawlers such as OAI-SearchBot, ChatGPT-User, Perplexity-User, and Claude-SearchBot visit web pages to retrieve real-time context when a user submits a prompt. These crawlers operate under strict latency constraints. They are designed to fetch, parse, and extract text from web pages as quickly as possible to populate prompt context windows.

When an answer crawler requests your page, it receives the raw HTML document returned by your server. If that document contains an iframe element pointing to a different origin, standard cross-origin security rules apply. In web browsers, a parent document cannot directly read or extract DOM nodes from an iframe on another domain unless specific cross-origin messaging setups exist.

AI answer crawlers do not run complete, unrestricted browser rendering engines for every nested sub-resource. Running full browser instances to execute JavaScript, evaluate iframe security policies, and wait for external dynamic content across millions of pages per hour is computationally prohibitive.

Instead, answer crawlers take a lightweight approach. They strip the DOM tree or parse the plain HTML text directly from the host response. To an answer crawler, an HTML tag like <iframe src="https://thirdparty-widget.com/table"></iframe> is evaluated as an empty element. The crawler extracts the surrounding copy on your main domain, ignores the external payload, and sends that truncated text back to the language model.

If your primary value proposition, pricing tiers, or feature tables live inside an external iframe, the language model never receives that text. It cannot summarize, evaluate, or cite information that never entered its context window.

Testing what an answer crawler sees with curl

You can verify how an answer crawler views your page by inspecting the raw server output directly from your terminal. Answer crawlers rely primarily on the initial HTML response delivered by your server.

Run this command in your terminal to inspect your page while impersonating OpenAI's search retrieval agent:

curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" -sL https://yourbrand.com/pricing | grep -i iframe

This command returns the raw HTML line containing your iframe tag. The copy, figures, and data stored inside the external URL will not appear in this output.

To view the complete text payload that an answer crawler extracts from your page, strip the HTML tags entirely from the HTTP response:

curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot)" -sL https://yourbrand.com/pricing | sed -e 's/<[^>]*>/ /g'

Examine the output carefully. If your key feature matrices, pricing numbers, or customer comparison points do not appear in this raw text stream, an answer crawler will not feed them to the model.

Distinguishing answer crawlers from training crawlers

When diagnosing why content fails to appear in AI search results, site owners frequently inspect their robots.txt files. It is vital to distinguish between training crawlers and answer crawlers during this process.

Training crawlers, such as GPTBot, ClaudeBot, and Google-Extended, scan the web to collect data for training future base models. Blocking a training crawler prevents your web content from entering future model weights, but it does not dictate what an engine retrieves during a live search query right now.

Answer crawlers, such as ChatGPT-User, Perplexity-User, OAI-SearchBot, Claude-SearchBot, and PerplexityBot, retrieve live web pages at the exact moment a prompt is executed. Blocking an answer crawler stops the engine from viewing your page content today.

In our crawler index updated on 2026-08-08, we evaluated a panel of 54,082 domains. Of these, 33,670 domains returned a readable robots.txt file. Within those readable domains, 5,497 block at least one AI crawler, and 2,576 block at least one answer crawler specifically.

Looking at individual crawlers across those 33,670 readable domains:

• GPTBot (training) is blocked by 5,080 domains.
• ClaudeBot (training) is blocked by 4,603 domains.
• Google-Extended (training) is blocked by 4,275 domains.
• PerplexityBot (search) is blocked by 2,147 domains.
• ChatGPT-User (search) is blocked by 2,074 domains.
• OAI-SearchBot (search) is blocked by 1,579 domains.
• Perplexity-User (search) is blocked by 1,402 domains.
• Claude-SearchBot (search) is blocked by 1,400 domains.
• Claude-User (search) is blocked by 1,384 domains.
• Googlebot (general) is blocked by 610 domains.

In a separate census conducted on 2026-08-07 covering 4,226 active Y Combinator companies from public data, 3,755 domains had readable robots.txt files, and 253 blocked at least one AI crawler.

Even if your host domain allows all AI crawlers, if your iframe content relies on an external third-party domain that blocks ChatGPT-User or OAI-SearchBot in its own robots.txt file, the answer crawler will fail to fetch the nested page.

What Standing can and cannot inspect

At Standing, we track whether AI engines recommend your business across ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews. We use a fixed prompt basket and query each engine five times per prompt to produce statistical confidence intervals. AI engines are non-deterministic, meaning a single test query proves very little. A score must always be expressed with its interval, such as a mention rate of 34, plus or minus 6, within a 95% confidence interval.

While Standing measures brand visibility across these platforms, Standing does not run interactive headless browser instances to click, scroll, or flatten cross-origin iframe components during structural diagnostics. Our system evaluates the accessible DOM and text payload that live answer crawlers extract. Standing cannot inspect dynamic interactions locked behind external frames that answer crawlers themselves ignore.

Because of these execution limits, automated tracking platforms cannot fully replace manual technical inspection. You must manually inspect your DOM source code to verify that your essential value propositions exist as native HTML text on your primary domain.

What not to buy to solve iframe indexing

Vendors sell a variety of plugins, edge-rendering scripts, and optimization packages that claim to make dynamic widgets readable to AI engines.

Do not buy expensive iframe-flattening plugins or proprietary dynamic rendering scripts designed specifically for AI indexing. Many of these vendors claim that adding structured JSON-LD schema markup or an llms.txt file will force answer crawlers to parse cross-origin iframes.

We have no evidence that schema markup or llms.txt files induce AI answer crawlers to bypass cross-origin frame boundaries or execute client-side JavaScript that they would otherwise drop. AI search engines process the readable text served directly in the response. Adding metadata tags around an empty cross-origin frame element does not supply the missing text to the engine.

If you rely on third-party tools for pricing grids, calculators, or review tables, render that content directly within your host page HTML. Request native