What is nondeterminism?

In most software, the same input always produces the same output. Language models break that assumption. Beyond the randomness introduced deliberately through temperature, the way models are run in production, batched together with other requests, spread across different hardware, using shortcuts for speed, introduces small variations that can change the output even at a setting meant to minimize randomness.

The practical consequence is that "the same prompt" is never a guarantee of "the same answer," and that includes which sources get cited. Two identical prompt basket runs, submitted minutes apart, can come back with a different set of citing sources purely from this effect, with nothing about the underlying content or the model's training having changed at all.

This is why single-run tests of AI visibility are unreliable by design, not by mistake. Nondeterminism sets a floor on how much any one measurement can be trusted, and the only way past that floor is repetition.