Technical Summary: Rethinking the Idiomaticity Decomposability Hypothesis

Problem Statement

The paper addresses the long-standing debate regarding the nature of idiomatic expressions, specifically the tension between the Idiom Decomposability Hypothesis (IDH) and usage-based accounts. The IDH posits that idioms vary in semantic decomposability (the extent to which constituent meanings contribute to the figurative whole) and that this property predicts syntactic flexibility (e.g., the ability to undergo passivization or modification). Conversely, usage-based theories argue that idiom behavior arises from distributional experience, such as frequency, predictability, and familiarity, rather than internal semantic structure.

A critical limitation in resolving this debate is the reliance on human decomposability ratings. These judgments are known to be gradient, task-dependent, and subject to inter-speaker variability, making it difficult to isolate whether idiom behavior stems from semantic structure or purely distributional exposure. The authors propose using contextualised language models as controlled distributional learners to investigate these relationships. By training models on large text corpora without explicit access to semantic role structures or acceptability judgments, the study aims to determine what aspects of idiom behavior can be learned from distributional exposure alone.

Methodology

1. Operationalising Decomposability

The authors propose a model-internal measure of decomposability grounded in three theoretical stipulations: semantic alignment, distributed meaning, and the gradient nature of the property.

• Semantic Alignment: They define decomposability as the degree of semantic alignment between an idiom in context and its figurative gloss (a paraphrase expressing the intended meaning).
• Token-Level Contribution: Using a bidirectional transformer encoder (e.g., BERT), they compute a sentence-level representation $e(s)$ for an idiom-in-context and $e(s_g)$ for the glossed sentence.
• Perturbation Analysis: To estimate the contribution of individual constituents, they employ a "leave-one-out" masking strategy. For each token $j$ in the idiom, they mask it to create $s_{(-j)}$ and compute the similarity between the masked representation and the gloss representation.
• Scoring: The contribution of token $j$ ($\Delta_j$) is defined as the absolute change in similarity when the token is masked: $\Delta_j = |S_{fig} - S^{(j)}_{mask}|$.
• Aggregation: Expression-level decomposability scores are derived by aggregating token-level contributions using mean, maximum, Gini dispersion, or entropy.

2. Operationalising Syntactic Flexibility

Syntactic flexibility is quantified as the Shannon entropy of an idiom's distribution across constructional types (base form, adverb insertion, adjective insertion, passivization, and action nominalization). Higher entropy indicates greater flexibility.

3. Experimental Setup

• Models: The study utilizes bidirectional transformer models (BERT-base/large, ModernBERT, and OLMo-2/3) to ensure context is encoded from both sides of the idiom.
• Datasets: Experiments draw on the IMPLI dataset (figurative sentences paired with glossed paraphrases) and the Bulkes and Tanner dataset (human decomposability ratings).
• Training Dynamics: To analyze learning over time, the authors track representations across 100 checkpoints during the pretraining of OLMo-2 and OLMo-3 (7B models). They examine the interaction between training steps and idiom properties: frequency, surprisal (predictability), and decomposability.

Key Results

1. Alignment with Human Judgments

Model-derived decomposability scores exhibit only a weak positive correlation with human decomposability ratings (e.g., $r \approx 0.24$ for BERT-large). While statistically significant, the effect size suggests that models capture only a partial overlap with human intuitions, indicating that the two operationalisations reflect distinct properties.

2. Decomposability vs. Syntactic Flexibility

Contrary to the IDH, the study finds no systematic positive relationship between model-derived decomposability and syntactic flexibility.

• Across models and layers, correlations are consistently small and frequently negative (maximum $r \approx -0.16$).
• This negative relationship is most pronounced for prepositional phrase (PP) idioms, where higher decomposability correlates with reduced flexibility. This is attributed to the morphosyntactic rigidity of PP constructions rather than semantic factors.
• Crucially, for verb phrase (VP) idioms—the primary domain where IDH is expected to hold—no significant relationship was found.

3. Usage-Based Factors

• Frequency: A robust negative relationship exists between frequency and decomposability. More frequent idioms are judged (by humans) and represented (by models) as less decomposable, supporting the view that high-frequency idioms are stored as holistic units.
• Predictability: Surprisal (predictability) and frequency both contribute to the stabilization of idiomatic representations during pretraining.

4. Training Dynamics

Analysis of pretraining checkpoints reveals that the emergence and stabilization of idiomatic representations are driven by surprisal and decomposability, not frequency alone.

• Interaction effects between training steps and idiom properties show that the influence of decomposability, surprisal, and frequency diminishes as training progresses.
• Decomposability shows the strongest training-dependent effect, indicating it plays a prominent role in shaping representations early in training, relative to frequency and surprisal.

Significance and Claims

The paper claims to offer a "clear challenge" to the IDH by demonstrating that decomposability is not a reliable, mechanism-neutral explanation for syntactic flexibility. The absence of a positive link between decomposability and flexibility, even in the theoretically most relevant domain (VP idioms), suggests that syntactic behavior cannot be solely explained by internal semantic structure.

Instead, the authors argue that distributional predictability (surprisal) and frequency play dominant roles. The study highlights that:

1. Frequency alone is insufficient to explain the emergence of idiomatic representations; surprisal and decomposability are necessary to account for stabilization dynamics.
2. Model-internal measures provide a principled way to disentangle factors confounded in human judgments, isolating what is learned from exposure alone.
3. The negative correlation between frequency and decomposability supports usage-based accounts where frequent idioms are treated as holistic, non-decomposable units.

The authors conclude that investigating internal representations allows for distinguishing between properties inherent to semantic structure and those emerging naturally from distributional learning, thereby helping to resolve confounded factors in linguistic theory.

Limitations

The authors acknowledge several limitations:

• The proposed decomposability metric is one of many possible operationalisations.
• The pretraining analysis relies on decomposability scores derived from BERT to predict dynamics in OLMo, which may introduce architecture-specific biases.
• The study is limited to English idioms and may not generalize to other languages.