💬 NLP

Improving Interpretability of Lexical Semantic Change with Neurobiological Features

This paper proposes a method to improve the interpretability of Lexical Semantic Change (LSC) by mapping contextualized word embeddings to a neurobiological feature space, allowing for both superior performance in estimating semantic shifts and the systematic discovery of specific types of meaning changes.

Original authors: Kohei Oda, Hiroya Takamura, Kiyoaki Shirai, Natthawut Kertkeidkachorn

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Kohei Oda, Hiroya Takamura, Kiyoaki Shirai, Natthawut Kertkeidkachorn

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at an old family photo album. You see a picture of your grandfather wearing a suit and a top hat. You know that "suit" means formal clothing. But if you look at a photo from 100 years ago, the word "cool" might have meant "chilly temperature," whereas today, if you call someone "cool," you’re giving them a compliment.

Words are like living creatures—they grow, they age, and they change their personalities over time. This process is called Lexical Semantic Change (LSC).

The Problem: The "Black Box" of Language

Computer scientists have known for a while how to use AI to detect that a word has changed. They use complex math to see if a word's "vibe" has shifted. However, there is a big problem: the AI is like a doctor who says, "Your patient is sick," but when you ask, "Why?" they just shrug and say, "The math says so."

The AI can tell you a word changed, but it can't tell you how. It can't say, "This word used to be about 'sound' and now it's about 'social status'." It just gives you a bunch of unreadable numbers.

The Solution: The "Sensory Map"

The researchers in this paper decided to give the AI a set of "senses." Instead of just using raw math, they mapped word meanings onto a Neurobiological Feature Space.

Think of this like giving the AI a color palette or a flavor profile. Instead of just saying "the word coffee changed," the AI can now say:

"The Taste intensity went up."
"The Smell intensity went up."
"The Vision intensity stayed the same."

They used 65 specific "features" (like Vision, Audition, Taste, and Emotion) based on how the human brain perceives the world. It’s like moving from a black-and-white map to a high-definition, multi-sensory GPS.

What They Discovered: The "Personality Shifts"

Because the AI could now "feel" the words, the researchers could see specific patterns of change. They used a technique called Sparse PCA—think of this as a highlighter tool that picks out only the most important changes and ignores the noise.

They discovered different "types" of word evolutions:

The "Sound" Shift: A word like bluegrass used to just be a type of grass (something you see), but it shifted into a type of music (something you hear).
The "Social" Shift: A word like warming shifted from just "getting hot" to being part of a massive social/political concept like "global warming."
The "Mood Swing" (Amelioration & Pejoration): This is when a word's "moral compass" changes.
- Amelioration is like a "glow-up": The word terrific used to mean "causing terror" (scary/bad), but it evolved into "amazing" (wonderful/good).
- Pejoration is like a "downfall": Words can pick up negative baggage over time, becoming more associated with pain or harm.

Why This Matters

By giving AI a way to "sense" language the way humans do, we aren't just teaching machines to read; we are teaching them to understand the nuance of human experience.

Instead of just seeing a list of changing words, we can now see the story of how our culture, our senses, and our emotions have evolved through the very words we speak.

Technical Summary: Improving Interpretability of Lexical Semantic Change with Neurobiological Features

1. Problem Statement

Lexical Semantic Change (LSC) refers to the phenomenon where the meanings of words evolve over time. While recent advancements in Natural Language Processing (NLP) have significantly improved the ability to estimate the degree of LSC (quantifying how much a word has changed) using contextualized embeddings like BERT, these models remain "black boxes." Existing interpretability methods—such as showing neighboring words or generating definition sentences—are often ambiguous, lack a systematic framework, and rely on natural language which can be subjective. There is a critical need for a method that can explain how and in what direction a word's meaning has shifted using a structured, quantitative, and multi-perspective framework.

2. Methodology

The authors propose a novel approach that maps the high-dimensional semantic space of BERT to a structured neurobiological feature space based on the "Binder features."

Binder Feature Space: The method utilizes 65 primitive neurobiological features (e.g., Vision, Audition, Taste, Happy, Harm) originally proposed by Binder et al. (2016). These features represent the intensity of specific sensory or emotional attributes of words.
Mapping Mechanism ( $\psi$ ): To bridge the gap between BERT and Binder spaces, the authors train a regression model ( $\psi$ $ψ$ ) that maps a 768-dimensional BERT vector ( $r_w$ $r_{w}$ ) to a 65-dimensional Binder vector ( $b_w$ $b_{w}$ ). They compare two architectures:
- Linear Transformation (LT): A simple linear mapping.
- Multilayer Perceptron (MLP): A deeper neural network.
LSC Detection: To quantify change between two time periods ( $t_1$ and $t_2$ ), the authors calculate the Average Pairwise Distance (APD) between the sets of Binder-mapped contextualized embeddings for each period.
LSC Type Analysis: To identify patterns of change, the authors compute an LSC vector (the difference between the mean Binder vectors of two periods) and apply Sparse Principal Component Analysis (Sparse PCA). This identifies which combinations of Binder features characterize specific "types" of semantic shifts.

3. Key Contributions

Introduction of Neurobiological Features to LSC: The first study to use structured sensory/emotional features to provide a systematic explanatory framework for semantic change.
Enhanced Interpretability: Moving beyond word-based explanations to a dimension-based system where each dimension has a clear, human-understandable meaning.
Discovery of New LSC Types: By combining the mapping method with Sparse PCA, the authors identified previously unnoted categories of semantic change.
Detection of Amelioration and Pejoration: A specialized application of the method to detect words that have acquired more positive (amelioration) or negative (pejoration) connotations.

4. Results

Performance in LSC Estimation: The mapping to Binder space, specifically using the Linear Transformation (LT) model, outperformed the baseline BERT space in predicting LSC scores (SemEval-2020 Task 1). Notably, the LT model achieved the best performance among all methods that do not utilize external knowledge (like dictionaries or WordNet).
LSC Type Identification: Sparse PCA successfully identified 10 distinct LSC types. Examples include:
- Transportation: Increases in Motion, Fast, and Path.
- Sound: Increases in Audition and Loud (e.g., the word "bluegrass" shifting from a color to a music genre).
- Social: Changes related to Consequential or Duration (e.g., "global warming").
Amelioration/Pejoration Validation: The method successfully ranked known words of amelioration (e.g., terrific) and pejoration (e.g., abuse, addiction, trauma) highly, demonstrating its ability to track sentiment-based semantic shifts.

5. Significance

This research shifts the focus of LSC studies from mere prediction to explanation. By grounding semantic shifts in neurobiological dimensions, the authors provide a bridge between computational linguistics and cognitive science. This approach allows researchers to not only detect that a word has changed but to categorize the change (e.g., "this word became more auditory" or "this word became more social"), offering a more granular and scientifically rigorous understanding of linguistic evolution.