Graded strength of comparative illusions is explained by Bayesian inference

Technical Summary: Graded Strength of Comparative Illusions Explained by Bayesian Inference

Problem
Comparative illusions (CIs) are linguistic phenomena where comprehenders systematically misperceive semantically anomalous sentences as acceptable (e.g., More students have been to Russia than I have). While the sentence is nonsensical because the than-clause fails to provide a comparable degree, comprehenders often judge it as natural. Previous research has proposed that CIs result from "noisy-channel" inference, where comprehenders use Bayesian reasoning to infer a speaker's intended meaning ($s_i$) from a corrupted signal ($s_p$). The posterior probability $p(s_i|s_p)$ is proportional to the prior probability of the intended meaning $p(s_i)$ and the likelihood of corruption $p(s_p|s_i)$.

However, prior empirical support for this theory (e.g., Zhang et al., 2025) had significant limitations:

1. It did not quantitatively model the full posterior distribution, failing to account for prior probabilities ($p(s_i)$).
2. It focused primarily on the most illusory CI types (pronominal subjects) and could not explain the graded strength of illusions found in other CI types (e.g., those with full noun phrase subjects).
3. It lacked a unified quantitative framework to predict how variations in morphosyntax (e.g., pronoun vs. noun phrase, singular vs. plural) affect illusion strength.

Methodology
The authors conducted two experiments and a statistical modeling phase to test the noisy-channel hypothesis quantitatively.

• Experiment 1 (Acceptability Judgment):

  • Participants: 494 native English speakers (a tenfold increase over previous studies).
  • Design: A 2x2 within-subjects manipulation of the than-clause subject: Form (Pronoun vs. Noun Phrase) and Number (Singular vs. Plural).
  • Task: Participants rated the naturalness of 30 critical CI sentences and control sentences on a 7-point Likert scale.
  • Goal: To establish a behavioral baseline of graded illusion strength across different CI types.

• Experiment 2 (Sentence Correction):

  • Participants: 199 native English speakers.
  • Task: Participants acted as editors, making the fewest possible edits to transform the anomalous CI sentences from Experiment 1 into plausible, meaningful sentences.
  • Analysis:
    • Qualitative: Corrected sentences were categorized into interpretation types (e.g., "event comparison," "individual comparison," "event negation").
    • Quantitative: The Damerau-Levenshtein distance (DLD) between the original anomalous sentence and the corrected sentence was calculated to serve as a proxy for the noise likelihood $p(s_p|s_i)$.

• Statistical Modeling:

  • Posterior Approximation: The authors constructed a quantitative approximation of the noisy-channel posterior $p(s_i|s_p)$ using Bayes' rule:
    $$\hat{p}(s_i|s_p) = \frac{\hat{p}(s_i)\hat{p}(s_p|s_i)}{\hat{p}(s_p)}$$
    • Priors ($\hat{p}(s_i)$) and Evidence ($\hat{p}(s_p)$): Estimated using sentence probabilities from two statistical language models (GPT-2 Small and OPT).
    • Likelihood ($\hat{p}(s_p|s_i)$): Estimated as proportional to $e^{-\text{DLD}}$, based on the edit distances from Experiment 2.
  • Linking Functions: Two hypotheses were tested regarding how the posterior influences acceptability:
    • Max Link ($f_{max}$): Acceptability is driven by the single most probable interpretation.
    • Mean Link ($f_{mean}$): Acceptability is driven by the average probability of multiple plausible interpretations.
  • Regression: Bayesian multilevel cumulative ordinal regression models were used to predict Experiment 1 acceptability ratings. The models included the noisy-channel posterior estimates as predictors, controlling for Syntactic Log-Odds Ratio (SLOR), trial order, and baseline control sentence acceptability.

Key Contributions

1. Replication and Expansion: The study replicates the core finding that CIs are rated as highly acceptable as felicitous controls but significantly expands the scope by demonstrating graded illusion strength. Specifically, singular pronoun subjects produce the strongest illusion, while plural noun phrases produce weaker illusions.
2. Quantitative Modeling of the Posterior: The authors provide the first quantitative approximation of the full noisy-channel posterior distribution for CIs, integrating statistical language model probabilities with human behavioral edit data.
3. Prediction of Graded Effects: The quantitative model successfully predicts the variability in illusion strength across different morphosyntactic conditions, a phenomenon unexplained by prior theories.
4. Mechanism of Integration: The study identifies that the mean linking function (aggregating multiple plausible interpretations) outperforms the max linking function (relying on the single most likely interpretation) in predicting human acceptability judgments.

Results

• Behavioral Findings: Experiment 1 confirmed that pronoun singular CIs are indistinguishable from controls in acceptability, while plural pronoun and noun phrase conditions show a measurable, graded decrease in acceptability.
• Correction Patterns: Experiment 2 revealed that ~80% of corrections were "plausible," with "event comparison" and "individual comparison" being the dominant interpretations.
• Model Performance:
  • The noisy-channel posterior estimates significantly predicted CI acceptability ratings over and above strong controls (SLOR, order, baseline).
  • The mean linking function ($f_{mean}$) provided the best fit, indicating that acceptability is influenced by the aggregate probability of multiple plausible interpretations, not just the single most likely one.
  • This result held across two different language models (GPT-2 Small and OPT) and two datasets (the current study and a public dataset from Zhang et al., 2025).
  • The max linking function ($f_{max}$) showed weaker predictive power and, when included with the mean function, often yielded a negative coefficient, suggesting that the "best" single interpretation is insufficient to explain the data.

Significance and Claims
The paper claims that these findings provide strong empirical support for noisy-channel inference as a unified computational-level theory of language comprehension. Specifically:

• Rationality of Illusions: The study argues that CIs are not merely failures of processing (e.g., "good-enough" processing or shallow heuristics) but reflect Bayes-optimal sentence interpretation. The comprehender's brain is functioning correctly by integrating noisy sensory input with top-down priors.
• Unified Theory: The results converge with evidence from other linguistic illusions and non-illusory contexts, suggesting that noisy-channel inference is a general mechanism for diverse language processing phenomena.
• Methodological Advance: The work demonstrates that acceptability ratings capture a rich space of probabilistic sentence processing, extending beyond binary grammaticality judgments. It validates the use of statistical language models combined with behavioral data to approximate human inferential processes.
• Distinction from Good-Enough Processing: While acknowledging that "good-enough" processing (algorithmic level) and noisy-channel inference (computational level) may be compatible, the authors emphasize that the graded nature of CI effects is best explained by the mathematical structure of Bayesian inference (the posterior probability distribution) rather than by constraints on cognitive effort alone.

The authors remain modest regarding the specific noise model used (DLD-based) and the exact temporal dynamics of interpretation, noting that future work is needed to refine the noise likelihood term and the linking functions between posterior probabilities and behavioral measures. However, they assert that the core finding—that graded illusion strength tracks the quantitative posterior probability of plausible alternatives—is a robust confirmation of the noisy-channel theory.