💬 NLP

Quantifying non deterministic drift in large language models

This paper empirically quantifies baseline behavioral drift in large language models by demonstrating that output variability persists even at zero temperature across different models and prompt types, establishing a systematic reference point for evaluating future mitigation strategies.

Original authors: Claire Nicholson

Published 2026-01-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Claire Nicholson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, creative robot assistant. You ask it the exact same question, "What's the weather like?" and you tell it, "Be as precise and boring as possible." You'd expect it to give you the exact same answer every single time, right?

This paper, written by Claire Nicholson, investigates what happens when you actually try that with modern AI robots (called Large Language Models or LLMs). The surprising finding is: Even when you tell the robot to be perfectly consistent, it often isn't.

Here is a simple breakdown of what the study found, using everyday analogies:

1. The "Ghost in the Machine" (Non-Determinism)

The researchers treated two different AI models like they were identical twins:

The Cloud Robot: A model called gpt-4o-mini that lives on a big server farm (like a restaurant kitchen you can't see).
The Local Robot: A model called llama3.1-8b that runs on a single computer in a lab (like a chef cooking in your own kitchen).

They asked these robots the same questions over and over again, with the settings turned to "maximum consistency" (Temperature 0.0).

The Result: The Cloud Robot gave a different answer about 1 out of every 4 times. The Local Robot was more consistent, but still changed its answer about 1 out of every 10 times.

The Analogy: Imagine asking a baker to make a chocolate cake using the exact same recipe and ingredients. You'd expect the cake to look identical every time. But in this study, the baker (the AI) sometimes adds a pinch more salt, or arranges the sprinkles differently, even though you told them to follow the recipe perfectly.

2. Why Does This Happen?

The paper suggests two main reasons for this "drift":

The Model Itself: Bigger, more complex models seem to be more "jittery" and less consistent than smaller ones.
The Kitchen (Infrastructure): For the Cloud Robot, the way the request travels through the internet, how the computer processes the data, or even how the server is organized can change the result. It's like if the baker's oven had a slight temperature fluctuation or if the mixing bowl was slightly different size each time, even if the recipe was the same.

3. The "Temperature" Knob

The researchers also tested what happens when they turned up the "Temperature" (a setting that makes AI more creative and random).

Low Temperature (0.0): The robot tries to be boring and consistent, but still slips up occasionally.
High Temperature (0.7): The robot goes wild. In this mode, every single answer was different. It was like asking the baker to make a cake, but telling them to "get creative." Suddenly, every cake looked completely different.

4. The "Word Count" and "Similarity" Check

How did they measure this? They didn't just read the answers; they used math to compare them.

Unique Fraction: They counted how many times the robot gave a totally new answer.
Jaccard Similarity: This is a fancy way of asking, "How many words are the same?"
- When the robot was "boring" (0.0), the answers were about 90% similar.
- When the robot was "creative" (0.7), the answers were less than 50% similar.

5. What This Means for You (The "Variance Budget")

The paper doesn't tell you how to fix this problem. Instead, it says: "You need to know this is happening before you try to fix it."

Think of it like a "Variance Budget."

If you are building a system that generates financial numbers, you might say, "I can only tolerate a 5% difference in the output."
If the AI drifts more than that, you know you need to intervene.
But first, you need to know what the "normal" drift looks like without any help. This paper provides that baseline map.

Summary

This study is a "measurement report." It didn't invent a new way to stop the robots from changing their minds. Instead, it measured exactly how much they change their minds when left alone.

The main takeaway: Even when you think you have a robot locked down to be 100% predictable, it's actually wiggling around a bit. If you rely on a single answer from an AI, you might be missing the fact that it could have said something slightly different the next time you asked. To get the full picture, you might need to ask the same question a few times and look at the average.

Technical Summary: Quantifying Non‑deterministic Drift in Large Language Models

Problem Statement

Large Language Models (LLMs) are increasingly deployed for critical tasks ranging from summarization to decision support. A fundamental expectation in these deployments is that identical prompts, when run under fixed "deterministic" configurations (e.g., temperature set to 0.0 and fixed seeds), should yield identical outputs. However, empirical evidence suggests this is not always the case.

The paper addresses the phenomenon of baseline behavioural drift: the variability observed when the same prompt is issued multiple times without stabilizing instructions or operator-level interventions. This drift arises from two primary sources:

Intrinsic Model Properties: Sampling algorithms and model scale.
Infrastructure Induced Nondeterminism: Serving infrastructure, including miscompiled sampling algorithms, floating-point non-associativity, and opaque API optimizations (e.g., dynamic batching, request queuing).

The core problem is the lack of systematic, operator-free quantification of this drift. Without a measured baseline, it is impossible to distinguish between inherent model variability and the effects of proposed mitigation strategies.

Methodology

This study is a measurement-focused experiment designed to quantify baseline drift under ordinary, operator-free conditions. It does not introduce new stabilization techniques or prompt engineering methods.

Experimental Setup

Models Tested:
- gpt‑4o‑mini: A smaller model served via a proprietary cloud API.
- llama3.1‑8b: An open-weight model running locally on a single NVIDIA A100 GPU using the vLLM engine.
Prompt Categories: Five distinct categories were used to test sensitivity:
1. Underspecified summaries
2. Underspecified advice
3. Light constraints
4. Style prompts
5. Hard constraints
Prompting Modes:
1. Exact Repeats: Identical prompts run multiple times.
2. Perturbed Inputs: Synonyms and minor lexical changes introduced while preserving meaning.
3. Reuse: Previous answers fed back as input for the next prompt.
Parameters: Experiments were conducted at temperatures of 0.0 and 0.7.
Volume: Each model/prompt combination was run 30 times (gapfill dataset) or 20 times (small battery). Runs with API errors or refusals were excluded.

Metrics

The study relies on lexical metrics to provide a conservative lower bound on drift, acknowledging that semantic metrics are needed for deeper analysis but are reserved for future work.

Unique Output Fraction: The ratio of distinct answers to total runs (approximating the probability of a new answer on a second call).
Average Pairwise Jaccard Similarity: Computed on tokenized outputs to measure lexical overlap.
Word Count Statistics: Mean and standard deviation of word counts across runs.

Key Results

The experiments revealed that nondeterminism persists even under settings theoretically designed to be deterministic (Temperature 0.0).

Baseline Variability at Temperature 0.0:
- gpt‑4o‑mini: Produced distinct outputs in approximately 24% of runs (Unique Fraction = 0.24) with an average Jaccard similarity of 0.89.
- llama3.1‑8b: Produced distinct outputs in approximately 9% of runs (Unique Fraction = 0.09) with an average Jaccard similarity of 0.97.
- Observation: The API-served model exhibited roughly 2.5 times more variability than the local open-weight model.
Impact of Perturbation and Reuse:
- Perturbed Inputs: Increased variability significantly (gpt‑4o‑mini: 0.57; llama3.1‑8b: 0.27).
- Reuse Mode: Reduced variability compared to exact repeats (gpt‑4o‑mini: 0.20; llama3.1‑8b: 0.10).
Impact of Temperature (0.7):
- Raising the temperature to 0.7 resulted in near-complete diversity. Almost every run yielded a new answer (Unique Fraction $\approx$ 1.0), and lexical similarity dropped below 0.52 across all modes.
Model and Deployment Differences:
- Larger models and API-served deployments generally exhibited higher variance. The study suggests that some drift in API environments reflects the inference stack (hidden optimizations, batching) rather than the model weights alone.

Key Contributions

Systematic Quantification of Baseline Drift: The paper provides the first systematic, multi-mode measurement of baseline behavioural drift across two distinct deployment types (API vs. local), five prompt categories, and three prompting modes.
Evidence of Persistent Nondeterminism: It demonstrates that "deterministic" settings (Temperature 0.0) do not guarantee reproducibility. Variability patterns differ by model size, deployment method, and prompting mode.
Conceptual Framework via Variance Budgets: The paper contextualizes these measurements using the concept of a "variance budget" (analogous to an error budget in reliability engineering). This frames drift not as a binary failure but as a resource allocation problem where applications must decide acceptable levels of variability (e.g., $\pm$ 5% for financial figures).
Separation of Measurement and Control: The study explicitly distinguishes between measuring drift and mitigating it. By establishing an unbiased baseline, it prevents the conflation of measurement noise with the efficacy of future stabilization techniques.

Significance and Claims

The paper positions itself as a descriptive, foundational study rather than a prescriptive solution. Its significance lies in:

Establishing a Reference Point: It provides a necessary baseline against which future mitigation studies (e.g., prompt engineering, constrained decoding) can be compared. Without this baseline, it is impossible to determine if an intervention genuinely reduces drift or merely masks it.
Highlighting Infrastructure Effects: The results suggest that a significant portion of observed drift in production environments stems from the inference stack (APIs, hardware, batching) rather than the model itself.
Defining the Limits of Lexical Metrics: While the study uses lexical metrics for interpretability, it explicitly acknowledges their limitations (e.g., flagging harmless paraphrasing as drift) and calls for the integration of semantic metrics in future work.
Implications for Reliability: The findings imply that relying on a single run of an LLM can obscure worst-case performance. Reporting multiple runs is recommended to provide a complete picture of model behavior.

The authors conclude that non-determinism is an inherent feature of current LLMs arising from both scale and infrastructure. Understanding and quantifying this drift is a prerequisite for improving the reliability of LLMs in multi-agent systems and long-horizon interactions.