What is sampling variance?
Sampling variance is the run-to-run variation you get from asking a language model the same question more than once.
It exists because these models are probabilistic by construction. Names appear in one answer and not the next with no change in the underlying world, no change in the sources, and no change in the question.
It is the largest single source of movement in AI visibility scores, and it is entirely noise. Distinguishing it from real change is the reason multiple runs and a confidence band are not optional refinements.
Related
- Confidence bandA confidence band is the range around a measured score that reflects how much of the number is sampling noise rather than signal.
- Model driftModel drift is when an engine's answers change because the provider changed the model behind it, not because anything about your brand changed.
- AI visibilityAI visibility is how often, and how prominently, a brand is named by AI assistants when people ask questions in its category.
Want to know where you actually stand on this? Run a free visibility check or try the free tools.