💬 NLP

Focus particles and scalar inferences across humans and language models

This paper investigates whether humans and large language models generate consistent scalar judgments for focus particles like "even" and "only," finding that while their outputs often align, they likely rely on fundamentally different underlying mechanisms.

Original authors: Catherine M. Brousse, Nelu D. Radpour

Published 2026-08-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Catherine M. Brousse, Nelu D. Radpour

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out how a brain—or a super-smart computer—decides what something means. Scientists have long wondered if our brains treat words like physical objects we can move around in space, or if they treat them more like emotional signals. Think of it like this: when you hear a word that means "good," do you automatically think "right" or "up," and when you hear "bad," do you think "left" or "down"? This idea is called "spatial coding," and it suggests our minds might be like a map where meaning is drawn with arrows. But there's another idea: maybe our brains just look at the feeling of the word (is it positive or negative?) and ignore the map entirely. This is the "evaluative" view. Understanding which one is true matters because it helps us figure out if artificial intelligence thinks like us, or if it's just a really good mimic that doesn't actually "feel" the way we do.

In this study, researchers Catherine Brousse and Nelu Radpour decided to put this to the test using two very different "brains": a group of 108 human volunteers and a massive language model called Llama 3.3 70B. They used a special kind of word trick called "focus particles." You know those little words like "even" and "only" that change the vibe of a sentence without changing the main facts? For example, saying "Mike can even bake a cake" sounds like baking a cake is a surprise or a low bar, while "Mike can only bake a cake" sounds like he has no other skills, which is a very specific, limiting judgment. The researchers wanted to see if changing the layout of the answer choices would change how people and the AI rated these sentences.

They set up four different ways to ask for answers. Imagine a slider for rating ability: sometimes it was a horizontal line (low on the left, high on the right), sometimes a vertical line (low on the bottom, high on the top). They also flipped the labels, so "low" was on the right or top, just to see if the brain got confused by the direction. Humans and the AI both read 60 sentences about people doing tasks, some with "even" and some with "only," and rated how capable the person seemed on a scale of 1 to 5.

The results were fascinating. Both humans and the AI agreed on the basic logic: sentences with "only" got high ratings (meaning the person seemed very capable in a specific way), and sentences with "even" got lower ratings. Crucially, it didn't matter if the answer buttons were flipped upside down or sideways. The judgments stayed the same. This suggests that for both humans and the AI, these word judgments are driven by the meaning of the words, not by where the buttons are on the screen. The "map" theory didn't win; the "meaning" theory did.

However, the AI and the humans weren't identical twins. The AI was much more extreme in its opinions. When the AI saw a sentence with "only," it gave the maximum score of 5 every single time, with zero variation. Humans, on the other hand, were more nuanced, giving a mix of high scores but not always the perfect 5. The researchers found that even when they tried to make the AI more "random" by changing its internal settings (called temperature), it still gave the same answer over and over. This suggests that while the AI can copy the general pattern of human thinking, it lacks the natural, messy variability that real people have. It's like the AI is a robot that memorized the rulebook perfectly, while humans are a crowd of people who all agree on the rule but still have their own little quirks. The study suggests that current AI models might be great at getting the "average" right, but they might not be the best models for understanding how real human minds actually work, because they don't seem to have that same range of individual differences.

Technical Summary: Focus Particles and Scalar Inferences Across Humans and Language Models

Problem and Motivation
The study addresses a central question in cognitive science and natural language processing: how humans and artificial systems interpret structured meaning, specifically regarding focus-sensitive particles like even and only. These particles modify the inferred meaning of a sentence by introducing expectations about alternatives (e.g., even highlights low-probability alternatives, while only introduces exclusivity) without altering core propositional content.

A key theoretical debate concerns the mechanism driving scalar judgments associated with these particles. One account posits that judgments are grounded in spatial coding, where abstract magnitudes are mapped onto spatial dimensions (e.g., left-right or up-down), implying that response scale orientation should systematically influence judgments (Dehaene et al., 1993). An alternative account, based on the polarity correspondence principle, suggests that judgments are driven by evaluative and linguistic structure, meaning differences between particles should remain stable regardless of spatial configuration (Proctor & Cho, 2006).

With recent evidence that Large Language Models (LLMs) develop structured internal representations of linguistic information (Kryvosheieva et al., 2025), the authors investigate whether LLMs exhibit human-like scalar judgment patterns. Crucially, unlike humans whose spatial representations are shaped by embodied interaction, transformer-based models process tokens simultaneously via bidirectional attention (Vaswani et al., 2017). This study tests whether similar scalar judgments emerge from shared representational principles or distinct underlying mechanisms.

Methodology
The researchers conducted a comparative study involving human participants and an LLM across four response-scale configurations.

Stimuli: 60 sentences describing ability-relevant tasks (e.g., "Mike can bake a cake"). Thirty items contained the particle even, and thirty contained only.
Human Participants: $N = 108$ participants completed one of four Qualtrics studies.
Experimental Manipulations: The spatial configuration of the response scale was manipulated along two dimensions:
1. Spatial Format: Horizontal vs. Vertical.
2. Label Mapping: Standard (low ability on left/bottom, high on right/top) vs. Reversed (labels flipped).
  This yielded four conditions: horizontal-standard, horizontal-reversed, vertical-standard, and vertical-reversed.
Procedure: Participants rated the ability of the individual in the sentence on a 5-point Likert scale.
LLM Data Collection: Data was collected using Llama 3.3 70B via the Groq API. Each stimulus was presented in a prompt mirroring the human survey structure. The model was instructed to output a single integer (1–5). To sample from the model's response distribution, each prompt was run 20 times at a temperature of 1.0, generating 4,800 observations (60 sentences $\times$ 4 conditions $\times$ 20 samples).

Results
Ratings were analyzed using a linear mixed-effects model with Particle, Source (Human vs. LLM), Format, and Mapping as fixed effects.

Particle Effects: A significant main effect of Particle was found ( $\beta= 2.20, p < .001$ ). Both humans and the LLM assigned lower ratings to even sentences and higher ratings to only sentences.
Source Differences: A significant Particle $\times$ $\times$ Source interaction ( $\beta= 1.38, p < .001$ $β = 1.38, p < .001$ ) revealed that the LLM produced more extreme judgments.
- Humans: Averages were 2.07 for even and 4.31 for only (Difference: 2.20).
- LLM: Averages were 1.41 for even and 5.00 for only (Difference: 3.58).
- Notably, for only sentences, the LLM assigned the maximum rating (5) in 100% of observations across all conditions.
Spatial Effects: Changing the orientation or label mapping of the response scale did not eliminate or reverse the pattern of judgments for either group. This suggests scalar judgments are driven by semantic and evaluative content rather than spatial arrangement.
Variability: The LLM exhibited substantially less response variability than humans. Even with a sampling temperature of 1.0, repeated sampling produced highly consistent responses. A follow-up temperature sweep (0.0 to 2.0) confirmed that model responses remained stable, indicating that increasing temperature alone was insufficient to generate human-like variability.

Key Contributions

Mechanism of Scalar Judgment: The study provides empirical support for an evaluative account of scalar judgments over a spatial coding account. The stability of judgments across spatial configurations suggests that the semantic structure of even and only drives interpretation more strongly than the spatial organization of response options.
LLM vs. Human Cognition: The paper demonstrates that while LLMs can reproduce aggregate human behavioral patterns (qualitative trends), they diverge significantly in response variability. The LLM's tendency toward categorical, extreme judgments contrasts with the graded, variable responses of humans.
Limitations of Sampling: The findings challenge the assumption that repeated sampling at high temperatures can simulate human individual differences. The authors argue that variability across model runs reflects noise around a fixed judgment, whereas human variability reflects genuine individual differences in interpretation.

Significance and Claims
The authors conclude that the lack of response variability in the LLM raises questions about the suitability of current LLMs as models of human cognition, given that individual differences are a defining feature of human behavior. The paper aligns with recent findings (Qiu et al., 2025; Rodriguez et al., 2026) that contemporary LLMs fail to reproduce the breadth of variability observed in human populations, even when average task performance is well-matched. The study modestly suggests that while LLMs capture the structural logic of scalar inferences, they do not yet model the stochastic nature of human judgment.

Technical Summary: Focus Particles and Scalar Inferences Across Humans and Language Models

More like this