💬 NLP

Losing My Composure: Predicting Compositionality Over Time

This paper introduces a novel dataset and task for predicting the diachronic trends of noun compound compositionality in German and English, revealing only a slight decrease in transparency over time and demonstrating that semantic models trained on narrow, temporally specific data slices outperform those trained on broad historical windows in capturing these gradual changes.

Original authors: Chris Jenkins, Emma Raimundo Schulz, Filip Miletić, Sabine Schulte im Walde

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chris Jenkins, Emma Raimundo Schulz, Filip Miletić, Sabine Schulte im Walde

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine language as a giant, bustling city where words are like buildings. Some buildings are simple shacks made of one brick; others are complex skyscrapers built by snapping two smaller buildings together, like a "coffee house" or a "freight train." For a long time, linguists had a hunch about these skyscrapers: they thought that as the city got older, these buildings would slowly lose their "blueprint." They believed that over centuries, a "coffee house" would stop looking like a house for coffee and start becoming a weird, mysterious unit that you just have to memorize, losing its connection to the words "coffee" and "house."

This paper is like a team of time-traveling detectives who decided to test that hunch. They didn't just look at two snapshots of the city (one from the past, one from today); instead, they built a high-speed camera that took pictures every single decade for hundreds of years. They focused on 23 German and 26 English compound words, like "bosom friend" or "Kaffeehaus," and asked human volunteers to rate, on a scale of 0 to 5, how much the meaning of the whole word still matched the meanings of its parts in each specific decade.

The Big Surprise: The Blueprint Stays Put
The detectives found that the old hunch was mostly wrong. They didn't find a massive, city-wide trend where these word-buildings were crumbling into mystery. In fact, the changes were so tiny they were almost invisible. If you measured the "transparency" of these words over time, the average change was a microscopic negative blip (about -0.016 for English and -0.001 for German). It's like checking the height of a skyscraper every ten years and finding it hasn't grown or shrunk by even a single grain of sand. The paper suggests that, contrary to popular belief, these compound words generally keep their "construction plans" intact; they don't necessarily become less logical or more idiomatic just because time passes.

The Time-Traveling Cameras
To make these measurements, the team built about 100 different "cameras" (computer models) to simulate how a word is understood in different eras. They tried two main types of cameras:

The "Context-Aware" Cameras (like Modern BERT): These are fancy, high-tech lenses that look at the words surrounding a target to understand its meaning, kind of like how you understand the word "bank" differently if it's next to "river" or "money."
The "Static" Cameras (like Word2Vec): These are simpler lenses that give a word one single, fixed definition based on its history, like a dictionary entry that never changes.

They also tested how much "historical data" to feed into these cameras. Some were fed a massive 50-year chunk of history (a "full" window), while others were fed just one decade at a time, or a sliding window that grew one decade at a time.

The Results: Less is More
Here is where the plot thickens. The team discovered that the "fancy" context-aware cameras didn't necessarily win the race. In fact, the simpler, static cameras performed just as well, and sometimes even better, at predicting how compositionality changed over time.

More importantly, the paper argues that the size of the history you feed the camera matters a lot. When they trained the models on a massive 50-year block of text (the "full" schedule), the models got confused and missed the subtle, decade-by-decade shifts. However, when they trained the models on narrow, single-decade slices (the "decades" schedule), the models aligned much better with the human ratings. It's as if trying to understand the mood of a city by reading a 50-year-old newspaper archive is less effective than reading the daily paper from exactly the day you are interested in. The paper suggests that for tracking these tiny, gradual changes, you need to zoom in on the specific decade, not blur it together with the past.

The Verdict
So, what's the final word? The paper doesn't claim to have solved the mystery of language forever. Instead, it suggests that the idea of compounds inevitably becoming "less compositional" over time is likely an exaggeration. The changes are there, but they are small and messy, not a straight line down.

The study also warns us that while fancy AI models are cool, they aren't always the best tool for historical detective work. Sometimes, a simpler model trained on a very specific slice of time (just one decade) is more accurate than a super-complex model trained on half a century of data. The authors measured these results using statistical scores (like R² values around 0.36 for the best English models), which shows a decent but not perfect match between the computer's guesses and human ratings.

In the end, the paper is a reminder that language is a journey, but for these specific word-buildings, the journey hasn't been a dramatic collapse into confusion. They've mostly stayed true to their original blueprints, decade after decade, proving that sometimes, the simplest way to look at the past is to look at it one year at a time.

Technical Summary: Losing My Composure: Predicting Compositionality Over Time

Problem Statement

This study investigates the semantic change of German and English noun compounds, specifically focusing on the evolution of their degree of compositionality (the transparency of the relationship between a compound's meaning and its constituents) over time. The authors challenge the prevailing hypothesis in lexical semantic change literature that compound nouns tend to become less compositional (more idiomatic) over time, evolving from transparent combinations to opaque units.

Existing work in Lexical Semantic Change Detection (LSCD) largely relies on a "two-era" paradigm (comparing two distinct time periods), which may obscure gradual changes. Furthermore, previous diachronic studies on compositionality have been limited by a lack of ground-truth data spanning multiple time steps. The paper posits that interesting semantic shifts do not necessarily occur at a single temporal dividing line but unfold gradually. Consequently, the authors introduce the Compositionality Trend Prediction (CTP) task, which aims to model and predict compositionality ratings across fine-grained temporal slices (decades) rather than aggregating data over large, undifferentiated time spans.

Methodology

Data and Annotation

The study utilizes two diachronic corpora:

German: The Deutsches Textarchiv (DTA), covering 1680–1920.
English: The Corpus of Historical American English (COHA), covering 1810–2000.

The authors selected 23 German and 26 English target noun compounds based on frequency stability and the presence of "inflection points" (decades where relative frequency changes significantly). For each target, they sampled in-context examples from up to five decades surrounding these inflection points.

Annotation: Human annotators rated the compositionality of each compound instance on a scale of 0–5 for each constituent (modifier and head).
Quality Control: Inter-annotator agreement was measured using Krippendorff's alpha, with post-hoc filtering to remove low-agreement annotators. The resulting dataset provides per-decade ratings and allows for the calculation of compositionality trends ( $\Delta$ ).

Experimental Setup

The authors trained approximately 100 models for each representation type, covering different 1–5 decade slices of the corpora. They evaluated three main experimental variables:

Representation Type:
- Contextual: Modern BERT (trained from scratch on historical data) and 2nd-order Random Indexing.
- Static: Word2Vec (CBOW and Skip-gram) and Pointwise Mutual Information (PMI) association lists.
Temporal Window Schedule:
- Full: A single model trained on a 5-decade window.
- Decades: Separate models trained on single decades.
- Sliding-Full: Incrementally expanding windows (e.g., [1940], [1940–1950], [1940–1960]).
Similarity Measures:
- PRT (Prototype): Cosine distance between averaged vectors.
- APD (Average Pairwise Difference): Average cosine distance between all pairs.
- JSD (Jensen-Shannon Divergence): Applied to clustered representations (only for contextual models).

Evaluation Metrics

The models were evaluated against the human annotations using two primary metrics:

Alignment ( $R^2$ ): Linear regression between predicted per-decade compositionality scores and annotated ratings.
Trend Prediction (Spearman's $\rho$ ): Correlation between the model's predicted ranking of compounds by degree of change and the annotated ranking of change ( $\Delta$ ).

Key Results

Empirical Observations on Compositionality Trends (RQ1)

Contrary to the hypothesis that compounds generally become less compositional over time, the annotated data reveals no decisive generalized trend.

German: Mean $\Delta$ was $-0.0012$.
English: Mean $\Delta$ was $-0.0161$.
The changes observed were extremely small, often smaller than the variance between consecutive decades. While specific compounds (e.g., Donnerwetter in German) showed shifts toward non-compositionality, the aggregate data does not support a universal rule of semantic drift toward opacity.

Computational Performance (RQ2)

The experiments addressed whether contextual models with fine-grained temporal data outperform static models with coarse-grained data.

Temporal Granularity: Models trained on narrower temporal windows (specifically the decades and sliding-full schedules) consistently outperformed those trained on the entire 5-decade full window. This suggests that aggregating data over long periods dilutes the specific semantic signals required for accurate compositionality prediction.
Representation Complexity: Static representations (Word2Vec, PMI) were competitive with, and in some cases outperformed, complex contextual representations (Modern BERT).
- In English, Modern BERT with decades schedules achieved the highest $R^2$ scores (0.363).
- In German, 2nd-order Random Indexing and Skip-gram performed best.
- The performance gain of BERT did not justify the computational cost of training from scratch on historical data.
Similarity Measures: Aggregation measures (PRT and APD) generally outperformed clustering-based measures (JSD). The clustering approach, particularly with BERT embeddings, yielded the poorest results.

Contributions

Dataset and Task: The introduction of a novel, fine-grained dataset of in-context compositionality ratings across multiple decades for German and English, enabling the Compositionality Trend Prediction (CTP) task.
Empirical Findings: A refutation of the "universal drift toward non-compositionality" hypothesis based on diachronic annotation, showing that trends are negligible and highly compound-specific.
Modeling Insights: Evidence that for time-sensitive semantic tasks, fine-grained temporal training windows are more critical than model complexity. The study demonstrates that simpler static models trained on narrow temporal slices can be as effective as, or better than, complex contextual models trained on aggregated data.

Significance and Claims

The paper claims that the CTP task successfully encodes temporal dependencies, proving that evaluating target terms within their own specific temporal windows ("on their own journey through the ages") yields better alignment with historical semantic shifts than broad, aggregated modeling.

The authors modestly conclude that while their methods can detect semantic changes in specific compounds, the task is inherently difficult due to the instability of constituent meanings over time. They caution against over-interpreting association lists without close reading, noting that modern biases can influence the interpretation of historical data. Ultimately, the study suggests that for modeling semantic change in small-data, historical scenarios, simpler representations with high temporal resolution are preferable to complex, aggregated models.