Technical Summary: D-Models and E-Models: Diversity–Stability Trade-offs in the Sampling Behavior of Large Language Models

Problem Statement

Large Language Models (LLMs) are increasingly deployed in web-scale applications such as code generation, search, and recommendation systems. In these contexts, the model's generation process is effectively a sampling operation from a distribution of potential items. While LLMs are known to approximate real-world distributions, a critical gap exists in understanding whether their fine-grained, step-by-step token sampling probabilities ($P_{token}$) faithfully align with the task-level target distribution ($P_{task}$).

Current research has identified deficiencies in LLMs' ability to perform random number generation or coin flipping, yet these studies are largely empirical and scenario-specific. There is a lack of systematic analysis regarding the mapping between the internal token-level distribution ($P_{token}$) and the task-level objective distribution ($P_{task}$), nor is there a clear understanding of how these internal mechanisms impact practical outcomes in complex tasks. The core question addressed is: Can LLMs effectively understand and adapt to different task objective distributions, and do their internal sampling behaviors exhibit distinct patterns that affect diversity and stability?

Methodology

The authors propose a probabilistic framework to analyze LLM sampling behavior by relating three key distributions:

1. $P_{task}$: The target probability distribution over a candidate set defined by the task (e.g., a specific distribution of numbers to generate).
2. $P_{token}$: The internal probability distribution over the vocabulary at each generation step, output by the model.
3. $P_{result}$: The empirical distribution of the final generated output.

Experimental Design

The study employs a multi-stage validation approach:

• Simulated Distribution Tasks: The authors designed discrete probability sampling tasks with explicitly defined $P_{task}$ (e.g., "Extreme Distribution" with one dominant token and "Flat Distribution" with uniform probabilities). Models were prompted to generate sequences of numbers strictly adhering to these distributions.
• Metrics:
  • e-score: Quantifies the extremeness of $P_{token}$ (average of the maximum probability at each step). High scores indicate deterministic behavior.
  • ATVD (Average Total Variation Distance): Measures the divergence between distributions ($P_{task}$ vs. $P_{token}$, $P_{task}$ vs. $P_{result}$, and $P_{token}$ vs. $P_{result}$).
  • ATVD-step: Measures step-wise alignment between $P_{token}$ and $P_{task}$.
• Downstream Task Evaluation: The identified model types were evaluated on Code Generation (HumanEval benchmark) and Recommendation (MovieLens-1M dataset).
  • Code: Measured via $\Delta_{pass}$ (diversity of successful solutions) and vector similarity.
  • Recommendation: Measured via Precision and "can-hit" rate (proportion of recommendations falling within a pre-defined candidate list).
• Internal Mechanism Analysis: The study investigated the effects of Temperature, Prior Bias (intrinsic numeric preferences), Layer-wise Probability Evolution, and a Quota Compensation Hypothesis (whether models adjust future logits based on past sampling errors).

Key Findings and Model Classification

Through controlled simulations and downstream evaluations, the authors identify a striking dichotomy in LLM behavior, classifying models into two distinct types:

1. D-Models (Deterministic Models)

• Examples: Qwen-2.5, Llama-3.1, DeepSeek-R1, DeepSeek-v3.
• Behavior: Exhibit high e-scores (often > 0.9), indicating that $P_{token}$ is highly concentrated on a single token regardless of the target distribution.
• Mechanism: They employ deterministic global planning. Even when a uniform output is required, they tend to fixate on a specific token, leading to large step-to-step fluctuations in $P_{token}$.
• Performance:
  • Code Generation: Perform well, generating diverse candidate solutions through minor perturbations of a deterministic core, resulting in high $\Delta_{pass}$.
  • Recommendation: Show lower "can-hit" rates, often deviating from the provided candidate list to generate diverse but potentially irrelevant items.

2. E-Models (Exploratory Models)

• Examples: Mistral-Small, Gemma-2, GPT-3.5-Turbo, GPT-4o.
• Behavior: Exhibit lower e-scores and $P_{token}$ distributions that more closely align with $P_{task}$.
• Mechanism: They demonstrate exploratory local planning. Their token probabilities are more stable and adapt to the task's required distribution, though they still show a bias toward higher concentration than strictly required.
• Performance:
  • Code Generation: Show lower $\Delta_{pass}$; their broader exploration in syntax-sensitive tasks often introduces semantic or execution errors.
  • Recommendation: Achieve higher "can-hit" rates and precision, as their generation remains more tightly aligned with the candidate set constraints.

Internal Characteristics

• Temperature Sensitivity: E-Models are highly sensitive to temperature changes, allowing for flexible control over diversity. D-Models maintain low-entropy sampling even at higher temperatures, showing resistance to distribution flattening.
• Prior Bias: Analysis confirms that deviations from $P_{task}$ are not caused by intrinsic numeric preferences (priors) in the models.
• Layer Evolution: In early layers, both model types show uniform probabilities. However, in the "up-layers" (near output), D-Models maintain extreme confidence, while E-Models gradually converge toward the task distribution.
• Quota Compensation: The hypothesis that D-Models compensate for sampling errors by adjusting future logits (to match $P_{task}$) was rejected. Correlation analysis showed no significant relationship between past sampling residuals and future probability adjustments, suggesting D-Models rely on a global plan rather than reactive compensation.

Key Contributions

1. Probabilistic View of Web Tasks: The paper formalizes web tasks as probabilistic sampling processes and validates the existence of two distinct model behaviors (D-Models and E-Models) based on the alignment between $P_{token}$ and $P_{task}$.
2. Task-Specific Trade-offs: The study reveals a systematic diversity–stability trade-off. D-Models favor diversity and deterministic refinement (beneficial for code generation) but struggle with strict constraints (harmful for recommendation). E-Models favor stability and constraint adherence (beneficial for recommendation) but may lack the controlled diversity needed for complex code exploration.
3. Mechanism Analysis: By probing internal properties (temperature sensitivity, layer-wise evolution, and compensation mechanisms), the paper provides causal insights into why these behavioral differences exist, moving beyond empirical observation to mechanistic understanding.

Significance and Claims

The authors claim that these findings offer foundational insights into the probabilistic sampling behavior of LLMs. The identification of D-Models and E-Models provides practical guidance for model selection and configuration in real-world web applications:

• For tasks requiring reliability and strict adherence to constraints (e.g., search, recommendation, conversational agents where hallucination of non-existent items is critical), E-Models are preferred.
• For tasks requiring diverse solution exploration and iterative refinement (e.g., code generation, creative writing), D-Models may be more effective.

The paper concludes that ensuring the sampling process is auditable and explainable is crucial for responsible AI development. By understanding whether a model operates as a D-Model or E-Model, practitioners can better balance diversity with reliability under real-world uncertainty.