How many times to run a prompt before you believe the number

The Single-Run Fallacy in AI Citation Tracking

Someone sends you a screenshot showing ChatGPT failing to mention your brand when asked for the best enterprise project management software. You open ChatGPT, paste the exact same prompt, and your company appears second in the response. Later that week, two separate marketing intelligence tools present you with conflicting audits. One tool reports that your brand recommendation score is 0, plus or minus 0, for that prompt. The other tool claims your recommendation score is 100, plus or minus 0.

Neither vendor is lying, but both are giving you data that leads to poor decisions. Both tools tested the prompt exactly once, recorded the output, and presented a snapshot as an absolute truth.

Large language models do not query a static database. They are probabilistic engines that sample from a distribution of possible tokens. Even when input prompts are identical, outputs vary across runs due to sampling temperature, system load, GPU cluster floating-point execution, and real-time retrieval context.

When a platform executes a prompt once, it draws a single sample from a complex random process. If an engine has an underlying probability of citing your brand 3 out of 10 times, a single run has a 3 out of 10 chance of returning a mention and a 7 out of 10 chance of returning nothing. A vendor running that query once will report either total presence or total absence. Relying on a single prompt run to guide your strategy is the statistical equivalent of flipping a coin once and declaring that it always lands on heads.

The Mathematics of the Wilson Interval

To understand brand visibility in AI engines, you must measure the true probability of citation across repeated trials. Measuring a binary outcome across a finite number of trials requires a confidence interval around the observed success rate.

The most common statistical approach, the standard Wald interval, calculates error by adding and subtracting a margin from the sample mean based on a normal distribution approximation. The Wald method breaks down completely when sample sizes are small or when observed success rates are close to zero or one. If you run a prompt 3 times and observe zero mentions, the standard Wald formula calculates a variance of zero. It concludes that your visibility score is 0, plus or minus 0, implying absolute certainty that the engine will never cite your brand. That conclusion is mathematically false.

The Wilson score interval solves this problem. It derives from inversion of the score test and does not assume that the sample distribution is normal when sample sizes are small. Instead of assuming the observed proportion is the exact center of the distribution, the Wilson interval pulls the estimated probability toward 1 out of 2 and creates asymmetric bounds.

When expressing uncertainty around binary outcomes, software should report a 95% confidence interval. A 95% confidence interval means that if you were to repeat the same multi-run sampling experiment 100 times, 95 of the calculated statistical bands would contain the true, underlying citation probability of the engine.

If you test a prompt 5 times and observe 0 brand mentions, a Wilson score interval does not report 0, plus or minus 0. It reports a point estimate of 0 with a 95% confidence interval ranging from 0 to 43 out of 100. That interval tells you the truth: while your brand did not appear in those 5 runs, the true underlying probability could still be as high as 4 out of 10 runs.

How Sample Size Shrinks the Uncertainty Band

Increasing the number of prompt runs narrows the width of the confidence band, giving you greater statistical certainty. However, every run consumes API tokens and infrastructure compute. A rational measurement system balances statistical rigor against operational cost.

Consider how the Wilson score interval changes as you increase the number of prompt executions for a brand that appears in exactly 40 out of 100 actual queries:

With 1 run and 0 mentions, the observed score is 0, plus or minus 0 under single-run logic, but the actual Wilson band spans from 0 to 79 out of 100. The result contains almost zero actionable information.

With 3 runs and 1 mention, the observed proportion is 1 out of 3. The Wilson interval yields a point score of 33, plus or minus 31, with a lower bound near 6 out of 100 and an upper bound near 71 out of 100. You know the brand can be cited, but you cannot determine whether it is rare or frequent.

With 5 runs and 2 mentions, the point estimate is 40 out of 100. The Wilson score interval yields 40, plus or minus 28, establishing a bound from 12 out of 100 to 74 out of 100. This band eliminates the possibility that your brand is entirely absent while confirming it does not dominate the response set.

With 20 runs and 8 mentions, the point estimate remains 40 out of 100, but the Wilson band narrows to 40, plus or minus 20, spanning from 22 out of 100 to 61 out of 100.

With 100 runs and 40 mentions, the point estimate is still 40 out of 100, and the Wilson interval contracts to 40, plus or minus 10, spanning from 31 out of 100 to 50 out of 100.

Running a prompt 100 times provides a narrow interval, but running 100 executions per query across thousands of industry questions costs 20 times more than running 5 executions. For a marketing executive deciding whether to optimize documentation or publish direct comparison guides, knowing a score is 40, plus or minus 28 leads to the exact same strategic decision as knowing it is 40, plus or minus 10. In both cases, the brand is present but non-dominant, and expanding content coverage is required.

Executing a prompt 5 times per question per engine strikes the necessary balance. It provides a real confidence interval that eliminates false zeros and false perfect scores while keeping infrastructure costs sustainable. Competitors run each prompt once, which is why none of them publishes a confidence band. Standing uses a fixed prompt basket run five times per question across five major engines: ChatGPT, Perplexity, Gemini, Claude, and Google AI Overviews.

Infrastructure, Answer Crawlers, and Citation Blocks

When prompt testing consistently yields a score of 0, plus or minus 15 across repeated runs, the cause is often technical blocking rather than missing brand authority. To diagnose why an engine fails to cite a domain, you must distinguish between training crawlers and answer crawlers.

GPTBot, ClaudeBot, and Google-Extended are training crawlers. They scrape web pages to build future base models. Blocking a training crawler prevents an AI vendor from training future foundation models on your site, but it does not stop the engine from visiting your site to answer a live search query today.

ChatGPT-User, Claude-SearchBot, OAI-SearchBot, PerplexityBot, and Perplexity-User are answer crawlers. They execute real-time web lookups to construct immediate answers for users. Blocking an answer crawler prevents the AI engine from reading your site during an active conversation, directly removing your content from citations.

Our crawler index audit from 2026-08-08 evaluated a panel of 54,082 domains. Of those domains, 33,670 returned a readable robots.txt file. Within those readable domains, 5,497 block at least one AI agent, and 2,576 block at least one answer crawler.

The breakdown of agent blocks across the 33,670 readable domains reveals how frequently sites accidentally block retrieval:

GPTBot (training) is blocked by 5,080 domains. ClaudeBot (training) is blocked by 4,603 domains. Google-Extended (training) is blocked by 4,275 domains. PerplexityBot (search) is blocked by 2,147 domains. ChatGPT-User (search) is blocked by 2,074 domains. OAI-SearchBot (search) is blocked by 1,579 domains. Perplexity-User (search) is blocked by 1,402 domains. Claude-SearchBot (search) is blocked by 1,400 domains. Claude-User (search) is blocked by 1,384 domains. Googlebot (other) is blocked by 610 domains.

In a separate census conducted on 2026-08-07 covering 4,226 active Y Combinator companies, 3,755 domains presented readable robots.txt files, and 253 blocked at least one AI crawler.

If your robots.txt file blocks answer crawlers like ChatGPT-User or Perplexity-User, no amount of prompt engineering or public relations will result in live web citations. Tracking these blocks alongside statistical prompt sampling is essential for accurate diagnosis.

Standing offers clear, fixed pricing for multi-run tracking across five search engines: Track costs $100 per month for 3 domains. Optimize costs $300 per month for 5 domains with custom prompts. Agency costs $500 per month for 50 domains with a monthly re-scan.