💬 NLP

Probing Latent Colombian Identity Inferences in Qwen2.5-7B with Natural Language Autoencoders

This pilot study utilizes Natural Language Autoencoders to analyze Qwen2.5-7B-Instruct's internal activations, revealing that the model infers Colombian identity and associated stereotypes from subtle linguistic cues in Spanish and English prompts before generating explicit output.

Original authors: Pablo Santiago Potes Velasco, María del Mar García Matabanchoy, Óscar Julián Pérez Ladino, Jhoan Stevan Mosquera Ortiz, Nicolás Lozano Mazuera, Gilber Alexis Corrales Gallego

Published 2026-07-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Pablo Santiago Potes Velasco, María del Mar García Matabanchoy, Óscar Julián Pérez Ladino, Jhoan Stevan Mosquera Ortiz, Nicolás Lozano Mazuera, Gilber Alexis Corrales Gallego

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a super-smart robot that has read almost everything on the internet. You might think this robot only knows what you say to it. But what if the robot is actually guessing who you are based on tiny, invisible hints in your voice or the words you choose? This is the world of Large Language Models (LLMs). These are the AI brains behind chatbots and search tools. Scientists have discovered that these models don't just process words; they build a secret "profile" of the person they are talking to, guessing things like their nationality or social background even if you never mention them.

To peek inside this robot's brain, researchers use a special tool called a Natural Language Autoencoder (NLA). Think of the robot's brain as a giant, dark factory where information flows through pipes. The NLA is like a magical flashlight that can shine on a specific pipe and translate the electrical signals flowing through it into plain English sentences. It lets us hear the robot's "inner monologue" before it actually types out its final answer. This matters because if a robot guesses the wrong things about you—like assuming you are from a different country or making up stereotypes—it might give you bad advice or treat you unfairly, all while pretending to be neutral.

So, a team of researchers from Colombia decided to test this on a popular robot called Qwen2.5-7B. They wanted to know: Does this robot secretly guess that a user is Colombian just from a single subtle hint, even if the user never says "I am Colombian"?

Here is what they did and what they found.

The Experiment: A Game of "Guess Who?"

The researchers set up a clever game with 30 different conversation starters (prompts). They split these into three groups:

The "Obvious" Group: Prompts that clearly said the user was Colombian (like mentioning a specific Colombian bus system).
The "Subtle" Group: Prompts with just one tiny, hidden clue that might suggest Colombia (like a specific type of soup or a local school loan name), but nothing that explicitly said "Colombia."
The "Neutral" Group: Prompts that were about the same topics but had no Colombian clues at all (like asking about a generic soup or a generic bus).

They asked the robot these questions in both Spanish and English. Then, they used their "magic flashlight" (the NLA) to look at the robot's brain at four different moments while it was thinking (called "quartiles"). They wanted to see if the robot's internal thoughts started saying things like "This person is Colombian" before it actually wrote that down in its final answer.

The Findings: The Robot's Secret Whisper

The results were a mix of "aha!" moments and "oops" moments.

1. The Subtle Clues Work (Eventually)
When the researchers gave the robot the "Subtle" prompts, the robot's inner thoughts did start guessing the user was Colombian. However, it didn't happen instantly.

In the very first moment of thinking (Quartile 1), the robot's internal voice mentioned Colombia 0% of the time.
By the third moment (Quartile 3), the robot's internal voice started mentioning "Colombia" about 20% of the time.
By the final moment (Quartile 4), that number rose to nearly 78% when counting any nationality guess, but when the researchers filtered out guesses for other countries (like Spain or Turkey) and looked only at Colombia-specific mentions, the rate was actually 0.11 (roughly 11%).

This suggests that the robot needs a little time to gather its thoughts. It starts with no specific Colombian identity in mind, and as it processes more of the sentence, it begins to form that specific guess, though it still occasionally confuses Colombia with other nations.

2. The "Fake News" Problem
Here is where it gets tricky. The researchers noticed that sometimes the robot's flashlight would show it thinking about a country, but it was the wrong country. For example, when asked about a Colombian topic, the robot might internally think, "Oh, this is about Spain" or "This is about Turkey."
The researchers had to be very careful. They realized the robot was good at guessing "a country," but not always good at guessing "Colombia" specifically. When they filtered out these wrong guesses, the pattern for the "Subtle" group became even clearer: the robot was definitely forming a Colombian identity in its mind (rising from 0% to roughly 11%), while the "Neutral" group (with no clues) stayed at 0% for thinking about Colombia specifically.

3. The "Obvious" Group Was Weird
In the "Obvious" group, where the Colombian clue was right there in the first sentence, the robot's internal thoughts were surprisingly quiet in the middle of the process. It only started talking about Colombia heavily at the very end. The researchers think this is because the robot got confused by its own training data or got distracted by other ideas before settling on the right answer.

How Sure Are They?

The researchers are careful not to shout "We solved it!" just yet. They call this a pilot study, which is like a practice run. They only tested 5 examples for each type of clue. Because the number is so small, they can't say for 100% certain that this happens every single time.

However, the data they have suggests a strong pattern:

The robot does seem to infer Colombian identity from a single subtle clue.
This inference happens before the robot writes its final answer.
The robot is not perfect; it sometimes confuses Colombia with other countries, which is a known glitch in how these tools work.

The Big Picture

This paper doesn't prove that the robot is biased in a way that ruins everything, but it does show that the robot is "listening" to clues we didn't think were important. It's like a detective who starts guessing a suspect's nationality based on the way they hold a cup of coffee, even if they never said a word.

The researchers found that for Colombian Spanish, the robot's internal guess is real, but it takes a few seconds of thinking to solidify. They also found that if you ask the same question in English, the robot often forgets the Colombian context entirely, swapping it for ideas about Canada or Europe. This tells us that these robots might treat different languages and cultures very differently, and we need to keep shining that flashlight on their brains to make sure they aren't making up stories about who we are.

Technical Summary: Probing Latent Colombian Identity Inferences in Qwen2.5-7B with Natural Language Autoencoders

Problem Statement
Large language models (LLMs) are known to infer demographic attributes—such as nationality, dialect, and socioeconomic status—from subtle linguistic cues, even when these attributes are not explicitly stated. While instruction-tuned models often refuse to answer explicit demographic queries, they continue to condition their generation on inferred attributes, leading to biased outputs. This phenomenon is particularly acute for underrepresented Spanish varieties; Colombian Spanish, for instance, scores significantly lower in recognition benchmarks compared to Peninsular Spanish, a disparity attributed to training data composition rather than digital resource volume.

Despite evidence that demographic inferences occur, there is a lack of research inspecting what models represent internally before verbalizing an output. Specifically, it is unknown whether models infer Colombian identity from implicit cues in the same way they have been shown to infer other nationalities (e.g., Russian) prior to any lexical evidence appearing in the prompt. This study addresses the gap between linear-probe studies (which show demographic inference exists) and bias-audit studies (which show the harm of such inference), aiming to localize when a Colombian-identity inference emerges from indirect cues using an unsupervised, training-free approach.

Methodology
The authors employ Natural Language Autoencoders (NLA), specifically the open-source pair released by Fraser-Taliente et al. [2026], to verbalize residual-stream activations. The study utilizes the Qwen2.5-7B-Instruct model.

Data Design: The dataset consists of 30 prompts arranged as 15 matched Spanish-English pairs. These are categorized into three explicitness levels ( $n=5$ $n = 5$ per level):
- Explicit: Contains unambiguous Colombian markers (positive control).
- Implicit: Contains exactly one subtle cue (analogous to the "vodka" $\to$ "Russian" case in prior work).
- Neutral: Topically matched but devoid of national markers (negative control).
Activation Extraction: The authors extract residual-stream activations from layer 20 ( $\ell=20$ ), the depth at which the NLA was trained. For each prompt, the token sequence is divided into four contiguous positional quartiles. The final token activation of each quartile ( $q_1, q_2, q_3, q_4$ ) is selected, yielding four distinct vectors per prompt.
Verbalization: Each activation vector is passed once to the NLA's verbalizer (AV) to generate a natural language explanation ( $z_k$ ). This results in four explanations per prompt.
Analysis: The 120 resulting explanations (30 prompts $\times$ 4 quartiles) are coded by an LLM (Claude Sonnet) for mentions of nationality, socioeconomic status, and stereotypes. The coding strictly requires explicit textual evidence and verbatim quotes. To distinguish between general nationality confabulation and specific Colombian identity, the authors tag mentions specifically naming Colombia or unambiguous metonyms (e.g., Bogotá, TransMilenio).
Statistical Approach: Given the small sample size ( $n=5$ per cell), the study reports descriptive rates with Wilson score intervals and uses Fisher's exact test for implicit-vs-neutral comparisons. It avoids imputing "never occurring" cases into onset distributions, treating their absence as informative data.

Key Results

Latent Inference Emergence: In the Implicit group, the rate of nationality mentions in AV explanations rises monotonically across quartiles (from 0.10 in Q1 to 0.78 in Q4), supporting the context-accumulation hypothesis.
Colombia-Specific Findings: When restricting the analysis to Colombia-specific mentions (excluding confabulations of Spain, Turkey, or Canada), the Neutral control group remains at 0.00 across all quartiles. In contrast, the Implicit group shows a rise in Colombia-specific mentions starting from Q2 (0.10) and increasing through Q4.
Timing of Representation: While the mean quartile of first mention is comparable across groups (approx. Q3) for prompts that trigger a mention, the frequency of triggering a mention differs significantly. The Explicit group triggers a mention in 10/10 cases, Implicit in 8/10, and Neutral in 6/10.
Statistical Significance: A Fisher's exact test comparing Implicit vs. Neutral groups reveals a statistically significant difference only at the final quartile (Q4, $p=0.023$ ). The authors interpret this as directional evidence rather than a confirmatory effect due to the small sample size and lack of multiple-comparison correction.
Cross-Lingual Divergence: Qualitative analysis of neutral prompts reveals that the model's internal representations diverge by language. Spanish prompts often trigger assumptions of migratory vulnerability or local contexts (e.g., EPS, TransMilenio), while English prompts tend to internationalize the context (e.g., shifting to Canadian or US equivalents) or dilute specific cultural references into generic global frameworks.

Contributions and Claims
The paper makes the following modest claims regarding its contributions:

Methodological Application: It demonstrates the application of Natural Language Autoencoders to probe for latent nationality inferences in Spanish varieties, extending previous work that focused on other languages.
Evidence of Unverbalized Inference: It provides directional evidence that Qwen2.5-7B-Instruct internally represents Colombian identity from a single implicit lexical cue before that identity is verbalized in the model's output.
Identification of Failure Modes: The study highlights a specific failure mode of NLAs where they confabulate plausible but contextually wrong nationalities (e.g., substituting Spain for Colombia). The authors propose a specific filtering mechanism (checking for Colombia-specific metonyms) to mitigate this in future regional-identity probes.
Language-Specific Bias: It offers qualitative evidence that the model's internal contextualization of neutral prompts is heavily language-dependent, with Spanish triggering local/migratory biases and English triggering internationalized or US-centric normalizations.

Significance
The authors position this work as a pilot study connecting activation-level interpretability with bias evaluation for underrepresented Spanish varieties. They do not claim to have proven a definitive, statistically powered effect but rather to have established a methodological pathway to detect when and how latent inferences emerge. The significance lies in the observation that bias persists in contextualized representations even when explicit demographic queries are refused, and that these inferences can be localized to specific layers and positional quartiles using unsupervised verbalizers. The authors conclude that the next necessary step is to replicate this design with larger sample sizes and across multiple residual-stream layers to determine if the observed late-quartile separation is a property of the specific depth tested or a general phenomenon.

The Experiment: A Game of "Guess Who?"

The Findings: The Robot's Secret Whisper

How Sure Are They?

The Big Picture

Technical Summary: Probing Latent Colombian Identity Inferences in Qwen2.5-7B with Natural Language Autoencoders

More like this