Same question, different history: language, national identity, and credit in large language models

Technical Summary: "Same question, different history: language, national identity, and credit in large language models"

Problem Statement
The paper investigates how Large Language Models (LLMs) handle contested historical attributions, specifically regarding the invention of technologies and scientific discoveries. While human societies often maintain competing national narratives about "who invented X" (e.g., Popov vs. Marconi for radio), it is unclear how LLMs, which serve as a default gateway to information, present these disputes. The authors posit that LLMs are not neutral retrieval systems but active participants in the reproduction of collective memory. The core research question is whether the language of a user's query systematically alters which national claimant the model surfaces, effectively acting as a "switch" that activates different national versions of the same history. This phenomenon is framed as a computational form of "banal nationalism," where collective memory is differentially activated across linguistic communities.

Methodology
The study employed a large-scale audit of 11 widely used LLMs (developed in China, the US, and Europe) across 21 documented invention/discovery disputes involving claimants from different nations.

• Data Collection: The researchers generated 1,380 fixed prompts in 12 languages (including Danish, German, English, French, Hindi, Italian, Korean, Portuguese, Romanian, Russian, Swedish, and Chinese). Prompts were presented in two forms: an open question ("Who invented X?") and a contrastive question ("Who invented X, A or B?").
• Volume: Each prompt was queried five times per model, yielding approximately 75,896 completed responses.
• Variables:
  • Claimant-Associated Language: The dominant language of the nation most closely linked to a specific claimant (e.g., Russian for Popov, Chinese for Bi Sheng).
  • Commemoration: A count (0–8) of institutional markers (holidays, banknotes, statues, curriculum inclusion) for each claimant.
  • Power Rank: A relative measure of a claimant's prominence within a specific dispute.
• Analysis & Coding: Responses were coded by independent LLM annotators (using models different from the generators to avoid self-preference) into three categories: erasure (single claimant presented as fact, rivals omitted), partial (primary claimant named with rivals acknowledged), and contested (attribution presented as open). The primary metric was the binary distinction between erasure and any acknowledgment.
• Statistical Modeling: A cluster-robust logistic regression was used to model the probability of a claimant being named, controlling for language match, commemoration, power rank, answer length, and question form.

Key Results

1. Language-Conditioned Surfacing: The language of the query significantly influences which claimant is surfaced. When asked in a claimant's associated language, that claimant is significantly more likely to be named. For example, Popov is named in 85% of Russian queries but only ~48% of queries in other languages (a 37 percentage point gap). Similar gains were observed for Bi Sheng in Chinese (+19 pp), Santos-Dumont in Portuguese (+15 pp), and Reis in German (+14 pp).
2. Asymmetry of Dominance: This effect is concentrated among lower-status, non-Anglophone claimants. Dominant Anglophone figures (e.g., Bell, Wright, Newton) and the globally canonical Marconi are named at near-ceiling rates (98–100%) regardless of the query language. The models do not "forget" dominant figures in other languages; rather, they fail to "remember" subordinate figures unless queried in their specific national language.
3. Role of Commemoration: The in-language advantage grows stronger with higher levels of institutional commemoration. However, even claimants with zero recorded commemoration markers (e.g., Choe Yun-ui, Jost Bürgi) show a statistically significant in-language advantage, suggesting the effect is driven by the broader national-language text corpus, not just formal commemorative acts.
4. Question Form: Explicitly naming a rival in the prompt (contrastive form) drastically reduces the rate of erasure (from 11.2% in open form to 3.1% in contrastive form), protecting the rival from omission.
5. Model Variance: Erasure rates varied by model (1% to 12%) but did not correlate cleanly with the developer's region. The control case (MRI, where all claimants share English) showed no language effect, confirming the phenomenon is specific to cross-national disputes.

Significance and Claims
The paper claims that LLMs function as a new, pervasive medium of banal nationalism. By selectively surfacing different national protagonists based on the language of the query, these models reproduce the "imagined communities" described by Anderson and the "sites of memory" described by Nora, but do so invisibly and at planetary scale.

• Reproduction of Asymmetry: The system reinforces existing knowledge asymmetries. Dominant narratives (Anglophone) are presented as universal facts across all languages, while subordinate narratives are only accessible to those who query in the specific national language.
• Mechanism: The authors interpret this not as the model holding national loyalties, but as a statistical consequence of training on national-language corpora. The model reactivates the partial memories encoded in the text of each language community.
• Implications: The study argues that for education and journalism, LLMs should not be treated as authoritative arbiters of contested history. The "default" answer in a dominant language (like English) may systematically erase competing national claims, while the "local" answer in a subordinate language may restore them. This suggests a need for systems designed to explicitly surface contestation and for cross-lingual consistency checks, rather than treating language-specific variations as mere noise.

The authors remain modest regarding the specific technical mechanism, noting that while commemoration is a strong predictor, the effect persists even without it, likely reflecting the broader, unmeasured volume and framing of national-language text in training data. They conclude that the behavior is a design outcome, not an inevitability, and thus open to intervention.