💬 NLP

Convergence in Science, Divergence in Religion: Calibrated Framing Differences Across Wikipedia's Language Editions

This paper introduces a calibrated framing distance metric to demonstrate that while scientific concepts are described with high consistency across Wikipedia's language editions, religious concepts exhibit the greatest divergence, even after accounting for language-family alignment differences and encoder biases.

Original authors: Hung-Hsuan Chen

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hung-Hsuan Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a library where every book about the same subject is written by a different group of people, in a different language, without ever consulting one another. This is Wikipedia. With editions in over 300 languages, it stands as one of humanity's largest collaborative projects, yet its chapters are not translations of a single master text. Instead, a Turkish editor, a Korean editor, and a Polish editor might all write about "pilgrimage" or "censorship," drawing from their own local sources, cultural histories, and community values. This creates a unique opportunity for researchers: if we look at how these different communities describe the exact same idea, can we see where their perspectives align and where they drift apart? This question sits at the intersection of language, culture, and technology. It matters because the text we write today often becomes the training data for the artificial intelligence systems of tomorrow. If those systems learn from a library where the definition of "democracy" or "sacrifice" shifts depending on the language, the machines they build might end up telling different versions of history to different people.

A researcher set out to measure this drift, not by counting how many facts are missing from one language versus another, but by measuring how differently the same facts are framed. They gathered 2,799 articles from twenty different languages, covering a wide range of topics. To ensure they were comparing apples to apples, they anchored their study to 150 specific concepts, such as "karma," "democracy," "quantum mechanics," and "refugee," using a global database to guarantee that every language edition was discussing the same core idea. They also included a set of neutral concepts, like chemical elements and basic colors, which serve as a control group. The goal was to see if the way people write about religion or politics differs more from one language to another than the way they write about settled science.

The researcher faced a significant hurdle: the computer tools used to compare text are not perfect. These tools, known as sentence encoders, translate words into mathematical points in a vast space to measure similarity. However, these tools sometimes work better for some pairs of languages than others. For instance, the tool might naturally think French and Spanish are closer than Hindi and Chinese, simply because of how it was trained, not because of any real cultural difference. If the researcher had just measured the raw distance between the articles, they might have mistaken these technical quirks for cultural gaps. To fix this, they created a "calibrated distance." They first measured how far apart the neutral, uncontroversial articles were for each pair of languages. This gave them a baseline, a kind of zero point that accounted for the tool's natural bias. Then, they subtracted this baseline from the distance of the actual topics they were studying. This adjustment stripped away the technical noise, leaving only the genuine differences in how the communities described their subjects.

The results revealed a clear and striking pattern. When it came to religion, the different language editions were significantly more divergent than the neutral baseline. Concepts like "sacrifice," "clergy," and "martyr" showed the widest gaps in description. Similarly, politically charged terms like "censorship" and "refugee" were described very differently across languages. In contrast, the articles on science and technology were the most aligned. Topics such as "evolution," "DNA," and "quantum mechanics" were described with remarkable consistency across all twenty languages, often even more closely than the neutral control concepts. This suggests that while scientific knowledge tends to converge into a shared global understanding, cultural and religious narratives remain deeply rooted in local contexts.

The study also found that this divergence is not spread evenly across all political topics. While "censorship" caused a rift, concepts like "democracy" and "human rights" were described quite similarly across languages. This indicates that the split is not about politics as a whole, but about specific, sensitive ideas that touch on local power dynamics and identity. The researcher tested their findings using three different computer models to ensure the results were not just an artifact of one specific tool. While the exact size of the differences changed slightly depending on the model, the ranking remained the same: religion was the most divergent domain, and science was the most aligned.

One of the most important takeaways is that language families do not tell the whole story. One might expect that languages from the same family, like French and Spanish, would frame concepts more similarly than distant languages like Hindi and Chinese. While this was true for the raw, uncalibrated data, the effect largely disappeared once the researcher adjusted for the technical biases of the computer tools. This means that the differences in framing are not simply a matter of linguistic heritage; they are genuine cultural choices made by the editors. The study also highlighted that the Wikipedia community for Chinese is unique. Because the mainland version of the site is blocked in China, the Chinese edition analyzed in this study reflects the perspectives of editors in Taiwan, Hong Kong, and overseas communities, rather than a single, unified national viewpoint.

Ultimately, this research provides a new way to look at the global knowledge base. It shows that while humanity agrees on the facts of the physical world, we still tell very different stories about our beliefs and our struggles. The data suggests that if artificial intelligence is trained on this material, it will likely inherit these divergences, potentially offering different historical or cultural narratives depending on the language a user speaks. By calibrating their measurements to remove technical noise, the researcher has provided a clearer map of where these differences lie, separating the artifacts of the machine from the genuine variations of human culture. The code and data from this study are now available for others to use, ensuring that future investigations into how we frame our world can be built on a foundation of clarity rather than confusion.

Technical Summary: Calibrated Framing Differences Across Wikipedia's Language Editions

Problem Statement
While previous computational studies of cross-lingual Wikipedia editions have focused on coverage gaps (i.e., which facts or articles exist in one edition but not another), this paper addresses the question of framing divergence. Specifically, it investigates how differently language communities describe the same concept when articles do exist in multiple editions. The author posits that language editions are written independently by distinct editor communities drawing on different cultural contexts, potentially leading to significant framing differences in culturally loaded domains (e.g., religion, politics) compared to settled scientific knowledge.

A critical methodological challenge identified is that multilingual sentence encoders (e.g., LaBSE, MPNet) do not align all language pairs equally well. Raw embedding distances between language versions of a concept reflect both genuine content differences and artifacts of encoder alignment quality. For instance, the paper notes that even for "calibration concepts" with stable cross-cultural denotations (e.g., chemical elements, numbers), the mean distance between language pairs varies by a factor of 3.6 (ranging from 0.107 for Persian–Indonesian to 0.389 for Hindi–Chinese). Furthermore, raw distances exhibit a language-family bias (smaller within families, larger across them). Without correcting for these encoder-specific and family-specific artifacts, analyses risk mistaking technical alignment noise for cultural divergence.

Methodology
The study analyzes 2,799 valid articles derived from 3,000 concept–language observations. The dataset spans:

Concepts: 150 concepts across four domains (Religion, Politics, Science/Tech, Pop Culture) with 30 concepts each, plus a 30-concept calibration set of minimally culture-loaded items (chemical elements, numbers, colors, animals, natural kinds).
Languages: 20 language editions from 15 families and nine scripts, selected to maximize diversity while ensuring support by the LaBSE encoder.
Data Extraction: Lead sections of articles were retrieved via the MediaWiki API. Stub articles and disambiguation pages were excluded.

Core Metric: Calibrated Distance
The paper introduces a baseline-adjusted metric called calibrated distance to isolate framing divergence from encoder alignment artifacts.

Raw Distance: For a concept $c$ in languages $i$ and $j$ , the raw distance $d(c, i, j)$ is calculated as $1 - \cos(v_{c,i}, v_{c,j})$ , where vectors are L2-normalized mean-pooled sentence embeddings.
Calibration Floor: For each language pair $(i, j)$ , a baseline $b(i, j)$ is computed as the mean raw distance across the 30 calibration concepts available in both languages.
Calibrated Distance: The final metric $e(c, i, j)$ is defined as:
$e(c, i, j) = d(c, i, j) - b(i, j)$
A positive value indicates the concept is framed more differently than the neutral baseline; a negative value indicates it is framed more similarly.

Significance Testing
To ensure results are not artifacts of chance or sampling, the author employs:

Permutation Tests: Domain labels are shuffled 10,000 times, with the calibration baseline recomputed in every iteration to avoid construction bias.
Bootstrap Analyses: Concept-node and language-node bootstraps assess the stability of rankings against variations in concept selection and language sampling.
Mantel Tests: Used to compare the geometric structures of distance matrices across different domains.
Encoder Robustness: The pipeline is replicated using two additional encoders (MPNet and CMLM) to distinguish data properties from encoder-specific behaviors.

Key Results

Necessity of Calibration: Raw distances show a strong language-family effect (0.177 within families vs. 0.217 across families). After calibration, this gap shrinks significantly (+0.005 within vs. +0.001 across), confirming that the baseline adjustment effectively removes encoder alignment artifacts.
Domain-Level Divergence (LaBSE):
- Religion: Significantly above the calibration baseline (+0.034, Holm-adjusted $p=0.007$ ).
- Science/Tech: Significantly below the baseline (−0.024, Holm-adjusted $p=0.030$ ), indicating higher alignment than even neutral concepts.
- Politics & Pop Culture: Do not differ significantly from the baseline as aggregate domains.
Concept-Level Patterns:
- Most Divergent: Ritual and institutional religious terms (sacrifice, clergy, martyr, temple) and specific politically sensitive concepts (censorship, refugee).
- Most Aligned: Settled scientific knowledge (evolution, DNA, quantum mechanics) and canonical ideals (democracy, human rights).
- Political Nuance: While the "Politics" domain averages to the baseline, divergence is concentrated in specific sensitive concepts (e.g., censorship), whereas "democracy" and "human rights" are highly aligned.
Encoder Robustness:
- Ranking Consistency: All three encoders (LaBSE, MPNet, CMLM) consistently rank Religion as the most divergent domain and Science/Tech as the least. Concept-level rankings correlate strongly across encoders (Spearman $\rho = 0.75–0.79$ ).
- Magnitude Discrepancy: The statistical significance of the positive religion effect is specific to LaBSE. Under MPNet and CMLM, the religion effect is positive but not statistically significant. Science/Tech remains significantly below the baseline across all three.

Significance and Claims
The paper claims to provide a methodological framework for measuring cross-lingual framing differences that distinguishes genuine content divergence from encoder artifacts. By releasing code, data, and per-language-pair calibration baselines, the author enables future studies to measure framing without conflating technical alignment issues with cultural differences.

The results suggest that encyclopedic divergence is not uniform across domains. Instead, it is concentrated in culturally loaded religious concepts and specific politically sensitive topics, while settled scientific knowledge and universal ideals remain remarkably consistent across language editions. This implies that for multilingual language models trained on Wikipedia, language-conditioned historical or cultural narratives may be more prevalent in religious and sensitive political domains than in scientific ones. The paper explicitly notes that these findings reflect the data-side origins of such behaviors, distinct from model post-training effects.

Limitations Acknowledged
The author notes that calibrated distance is a relative measure, not an absolute ground truth. It cannot distinguish between genuine disagreement on meaning versus different emphasis on facets of a concept. Additionally, the "Chinese" edition reflects a transnational community (Taiwan, Hong Kong, overseas) due to access restrictions in the PRC, and the study is limited to lead sections. Finally, the significant positive religion effect is encoder-dependent (observed only with LaBSE), suggesting that absolute magnitudes should not be generalized across different embedding models.

Technical Summary: Calibrated Framing Differences Across Wikipedia's Language Editions

More like this