Technical Summary: Word-Centered Semantic Graphs for Interpretable Diachronic Sense Tracking

Problem Statement

Semantic shift—the transformation of word meanings over time due to cultural, technological, and social factors—is a central challenge in historical linguistics and natural language processing (NLP). While recent computational approaches have advanced the detection of semantic change, existing methods face significant limitations in representing polysemy (words with multiple meanings).

• Static Embedding Approaches: Methods relying on static word embeddings (e.g., Word2Vec) compress multiple senses into a single vector. This aggregation makes it difficult to disentangle sense-specific changes, as the model cannot distinguish between a shift in the dominant sense versus the emergence of a new one.
• Contextual Embedding Approaches: While contextual models (e.g., BERT, RoBERTa) can represent multiple senses, they typically require aggregating token representations into discrete sense inventories via clustering or supervised classification. This introduces challenges regarding comparability across time, sensitivity to the number of clusters, and the conflation of usage similarity with semantic relatedness.
• Interpretability Gap: Many existing methods rely on predefined sense inventories (like WordNet) or produce "black box" vector drifts that lack transparent, structural explanations for how senses evolve.

The authors argue that there is a need for a framework that represents word meaning relationally, captures the dynamics of polysemy without predefined inventories, and offers interpretability at both the sense and word levels.

Methodology

The paper proposes a word-centered semantic graph framework that integrates static distributional similarity with contextual lexical substitutability to model sense evolution. The methodology proceeds in four main stages:

1. Construction of Word-Centered Semantic Graphs

For a target word $w$ in a specific time slice $t$, the authors construct a graph $G^t_w = (N^t_w, E^t_w)$ by integrating two complementary sources of semantic evidence:

• Distributional Neighbors (Static): Using a Skip-gram (Word2Vec) model trained on the time-slice corpus, the top $k_i$ words with the highest cosine similarity to $w$ are selected. These capture global lexical associations.
• Substitution Neighbors (Contextual): A RoBERTa masked language model, fine-tuned on the same corpus, predicts substitutes for masked occurrences of $w$. The top $k_c$ predicted substitutes form the substitution set, capturing usage-conditioned meanings.
• Graph Expansion: The initial neighborhood is the union of these two sets. The graph is then expanded layer-by-layer (up to depth $L$) to capture indirect associations. Edges are added between nodes if they share distributional or substitutional relationships.

2. Sense Community Identification via Peripheral Connectivity

To identify distinct senses without clustering embedding vectors directly, the authors analyze the peripheral graph ($G^t_{w,per}$), which is the original graph with the central target node $w$ and its incident edges removed.

• Connected Components: The connected components of this peripheral graph are defined as sense communities ($C^t(w)$).
• Rationale: Since the central node trivially connects all neighbors, removing it reveals the underlying structure where neighbors cluster based on mutual semantic relatedness. Disconnected components in the peripheral graph represent distinct, co-existing senses (polysemy), while a single large component suggests a unified meaning.

3. Temporal Alignment and Refinement

To track senses across time, the framework aligns clusters from time $t$ with those from $t-1$ (or earlier periods) based on node overlap:

• Alignment Strategy: A cluster $C^t_i$ is aligned to the cluster $C^{t-1}_j$ that maximizes the intersection of their node sets.
• Historical Alignment: To handle re-emerging senses, alignment can also look back to any previous time slice $k < t$.
• Refinement: Clusters that do not persist for at least two time periods are merged into a residual cluster to filter out ephemeral noise or transient usage spikes.

4. Sense Usage Distribution Estimation

The frequency of a sense is estimated by the normalized cluster mass. The size of a cluster (number of nodes) relative to the total graph size provides a probability distribution over senses ($P^t(C_i | w)$), allowing for the tracking of sense dominance, broadening, or narrowing over time.

Key Contributions

1. Unified Graph-Based Representation: The paper introduces a novel framework that combines static distributional similarity and contextual substitutability into a single graph structure, avoiding the pitfalls of purely vector-based or purely inventory-based approaches.
2. Interpretable Sense Tracking: By defining senses as connected components in a peripheral graph, the method provides a transparent, structural explanation for polysemy and semantic change without relying on predefined sense inventories.
3. Diachronic Alignment without Vector Space Alignment: The approach tracks sense continuity through node overlap rather than requiring complex alignment of embedding spaces across different time periods.
4. Granular Analysis of Semantic Shift: The framework distinguishes between different types of semantic change, including event-driven sense replacement, gradual association shifts, and semantic stability.

Results and Application Study

The framework was applied to a diachronic corpus of New York Times Magazine articles (1980–2017), focusing on three target words with contrasting semantic profiles: trump, god, and post.

• Trump (Event-Driven Replacement):
  • 1980s: The graph showed a dense, cohesive structure corresponding to the literal "card game" sense (low polysemy).
  • 1990s–2000s: The graph expanded and fragmented, revealing disconnected communities related to business and real estate (Donald Trump's career), indicating a period of high polysemy.
  • 2015–2017: The graph underwent a substantive reconfiguration. The literal card-game sense disappeared from the neighborhood, replaced entirely by a political community (candidate, campaign). This demonstrated a complete sense replacement driven by historical events.
• God (Semantic Stability):
  • The graph remained structurally consistent over time with high inter-node similarities and no emergence of disconnected communities.
  • Peripheral clustering yielded many small clusters, but these were semantically homogeneous and highly interconnected, suggesting they represent facets of a single underlying sense rather than distinct meanings. This highlights the method's ability to distinguish between true polysemy and over-segmentation.
• Post (Gradual Association Shift):
  • The word exhibited stable neighborhoods until roughly 2005–2010.
  • Subsequently, the graph expanded to include nodes related to digital communication (e.g., twitter, video, email). Unlike trump, this was not a replacement of an old sense but a broadening of associations as technology became integrated into everyday language.

Significance and Claims

The authors position this work as a step toward interpretable and transparent semantic analysis.

• Interpretability: Unlike "black box" vector drifts, the graph structure allows researchers to visually and structurally inspect how senses emerge, coexist, and disappear.
• No Predefined Inventories: The method allows sense structures to emerge dynamically from corpus evidence, avoiding the bias of fixed dictionaries or the sensitivity of clustering hyperparameters.
• Polysemy Dynamics: The framework successfully captures the "polysemy dynamics" of words, showing how connectivity patterns (dense vs. disconnected) reflect the degree of polysemy.
• Modest Scope: The paper explicitly notes that this is an application study rather than a benchmark evaluation. It emphasizes the interpretability and qualitative insights gained from the framework rather than claiming superior quantitative performance on standard detection benchmarks. The authors acknowledge limitations regarding sensitivity to hyperparameters and potential corpus biases (e.g., the NYT Magazine editorial focus).

In conclusion, the paper demonstrates that word-centered semantic graphs offer a compact and flexible representation for exploring sense evolution, successfully modeling contrasting trajectories of semantic change without relying on external sense inventories.