Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora

This paper demonstrates that despite challenges with cultural nuances and slang, LLM-based machine translation effectively preserves subtle moral cues in Polish social media data, enabling cost-effective cross-lingual moral values research through high semantic similarity and minimal performance gaps in downstream classification tasks.

The Gist

Imagine you have a giant library of books written in English that teach a computer how to understand human morality—what makes us feel things like "care," "fairness," or "loyalty." Now, imagine you want to teach that same computer to understand these feelings in Polish, but you don't have any Polish books to start with.

In short: You don't need a perfect translation to understand the moral heart of a message. A good-enough translation, powered by modern AI, is enough to let computers learn about human values in new languages.

The usual solution would be to hire a team of human experts to read every English book, translate it, and re-label it in Polish. But that's expensive and slow.

This paper asks a simpler question: Can we just use a super-smart AI translator to do the job?

The author, Maciej Skórski (affiliated with the University of Warsaw), was worried because moral language is tricky. It's full of sarcasm, slang, inside jokes, and cultural references. It's like trying to translate a stand-up comedy routine; if you translate the words literally, the joke (and the moral point) often dies.

The Experiment: A "Moral Bridge"

To test this, the researcher took about 50,000 English social media posts (from Reddit and Twitter) that were already labeled with moral themes. He used a powerful AI (Claude Sonnet) to translate them into Polish.

Think of this translation process like building a bridge across a river. The river is the gap between English and Polish moral understanding. The question was: Will the bridge hold up under the weight of complex human emotions, or will it crumble?

The Safety Checks

The author didn't just trust the AI blindly. He set up four different "safety inspectors" to check the quality of the bridge:

1. The "Vibe Check" (LLM-as-Judge): Another AI read the translations and scored them on a scale of 0 to 10, looking for lost jokes, bad slang, or awkward phrasing.
   • Result: The translations got a 9.1 out of 10. They were mostly perfect, though some very specific slang (like African American Vernacular English on Twitter) was a little harder to translate perfectly.
2. The "Fingerprint Match" (Embedding Similarity): The computer looked at the mathematical "shape" of the sentences in English and compared it to the Polish. If the shapes are similar, the meaning is preserved.
   • Result: The shapes matched 86% to 89% of the time. That's a very strong match, meaning the core "feeling" of the sentence survived the trip.
3. The "Structural Integrity" Test (CKA): This checked if the overall map of the language stayed the same, not just individual sentences.
   • Result: The map held up well, confirming the translation didn't scramble the moral landscape.
4. The "Test Drive" (Classifier Parity): The researcher trained a computer to spot moral themes using the English texts, then tried to do the same with the Polish translations.
   • Result: The computer performed almost identically on both languages. The difference in success rate was tiny (only 1–2%), and when they tweaked the computer's settings (fine-tuning), the gap almost disappeared completely.

The Verdict

The paper concludes that moral semantics survive machine translation.

Even though the AI translator isn't perfect (it sometimes struggles with heavy slang or very specific cultural idioms), it preserves the "moral soul" of the text well enough for computers to learn from it.

Why This Matters (According to the Paper)

• It's Cheap: Translating 50,000 posts cost about $200. This is a fraction of the cost of hiring human translators.
• It Works for Polish: Polish is a very complex language with many grammatical cases (like a language with many different "outfits" for every word). If the bridge holds for Polish, the author suggests it will likely hold for other related Slavic languages too.
• It Opens the Door: This means researchers can now study moral discussions in Polish (and potentially other languages) without needing to wait for expensive, manually created datasets.

In short: You don't need a perfect translation to understand the moral heart of a message. A good-enough translation, powered by modern AI, is enough to let computers learn about human values in new languages.

Technical Summary

Technical Summary: Moral Semantics Survive Machine Translation

1. Problem Statement

Automated moral values classification relies heavily on language-specific annotated corpora, which currently exist almost exclusively in English. This creates a significant barrier for analyzing moral discourse in under-resourced languages, particularly within the Slavic family. While Machine Translation (MT) offers a potential shortcut to bridge this gap, moral language presents unique challenges: it is subtle, culturally variable, and laden with irony, slang, and register shifts that often resist literal mapping. Previous research suggests MT can inject systematic bias and distort affective signals. The core question addressed is whether Large Language Model (LLM)-based translation can preserve the moral-semantic content of Moral Foundations Theory (MFT) annotated texts well enough to support cross-lingual machine learning, specifically testing the translation of English to Polish.

2. Methodology

The study employs a principled, four-method validation pipeline to evaluate the fidelity of English-to-Polish translations of approximately 50,000 morally annotated social media posts.

2.1 Data and Translation Pipeline

• Corpora: Two diverse datasets were utilized:
  • MFRC (Moral Foundations Reddit Corpus): ~17.9k posts covering everyday morality, US politics, and French politics, characterized by colloquial language and abbreviations.
  • MFTC (Moral Foundations Twitter Corpus): ~33.9k posts covering political movements, civil unrest, and hate speech (Davidson subcorpus), featuring AAVE expressions, vulgarity, and activist hashtags.
• Translation Engine: Translations were performed using Claude Sonnet via the Anthropic API.
• Prompt Engineering: Two distinct system prompts were engineered for Reddit and Twitter to handle platform-specific conventions (e.g., preserving NTA/YTA abbreviations on Reddit; maintaining hashtags and @mentions on Twitter). The prompts explicitly instructed the model to preserve tone, translate slang/profanity into natural Polish equivalents (e.g., wtf → kurwa), and handle grammar/declension correctly while keeping proper nouns and formatting intact.
• Cost: The entire translation of ~50k posts cost approximately $200 USD.

2.2 Validation Framework

The pipeline validates translation quality through four complementary methods:

1. LLM-as-Judge: A stratified sample (~200 posts per subcorpus) was evaluated by an LLM on a 0–10 scale across four dimensions: tone preservation, slang handling, formatting fidelity, and proper noun treatment.
2. Embedding Similarity (LaBSE): Cross-lingual cosine similarity was computed between English source and Polish translation sentences using LaBSE embeddings.
3. Centered Kernel Alignment (CKA): This metric measured the global alignment between the embedding spaces of the source and target languages, capturing structural preservation beyond pairwise distances.
4. Classifier Parity Tests: A linear classification head was trained on frozen LaBSE embeddings (10-fold cross-validation) to compare ROC-AUC scores between English and Polish conditions. Additionally, mDeBERTa-v3 was fully fine-tuned on both English and translated Polish corpora to confirm parity under full gradient updates.

3. Key Results

3.1 Translation Quality (LLM-as-Judge)

• Overall Score: The mean score across all subcorpora was 9.1/10.
• Cleanliness: 94.6% of posts were free of detectable issues.
• Challenges: Scores were slightly lower (8.5) on subcorpora heavy in AAVE (e.g., All Lives Matter, Davidson) and Baltimore uprising, where dialect and embedded lyrics occasionally necessitated paraphrasing.
• Failure Modes: Persistent minor issues included sporadic translation of hashtag content and occasional Cyrillic character leakage during self-correction.

3.2 Semantic Similarity

• Cosine Similarity: The mean cross-lingual cosine similarity was 0.889 (MFRC: 0.876; MFTC: 0.894), well above the random baseline (~0.30) and exceeding the 0.80 threshold for strong semantic equivalence.
• CKA: The global embedding alignment score was 0.860. The lowest score (0.806) occurred in the Davidson (hate speech) subcorpus, attributed to AAVE paraphrase shifting distributional geometry.

3.3 Classifier Parity (Downstream Utility)

• Frozen Embeddings: ROC-AUC gaps between English and Polish were minimal, ranging from 0.01 to 0.02 across most foundations.
  • The Authority foundation showed the most consistent, though minor, degradation (gaps 0.021–0.031).
  • Fairness and Sanctity showed no significant degradation.
  • In the Davidson corpus, gaps were slightly larger (up to 0.048) but did not invalidate the data for downstream use.
• Full Fine-Tuning: When mDeBERTa-v3 was fully fine-tuned, the English and Polish models followed near-identical learning trajectories. In the Davidson corpus, both converged from ~0.57 to ~0.68 ROC-AUC, with a final gap of only 0.006.

4. Contributions

1. Principled Validation Framework: A reproducible, multi-method pipeline combining LLM-as-judge evaluation, embedding similarity (LaBSE, CKA), and deep learning classifier parity tests, applicable to any source/target language pair.
2. Large Translated Corpus: A validated English-to-Polish translation of 50k morally annotated social media posts, produced at a low cost ($200), demonstrating the accessibility of this approach for research groups without large annotation budgets.
3. Empirical Evidence of Generalizability: The study demonstrates that despite the morphological complexity of Polish (a demanding test case for Slavic languages), machine translation preserves subtle moral cues sufficiently for cross-lingual machine learning. This suggests the approach generalizes to the broader Slavic language family.

5. Significance and Claims

The paper claims that moral semantics survive machine translation. Contrary to concerns that MT distorts the nuanced, culturally loaded nature of moral language, the results indicate that LLM-based translation is a practical and cost-effective path for moral values research in under-resourced languages.

• Robustness: Moral language, despite its irony and idiom, proves more robust to translation than previous literature on LLM limitations suggested.
• Feasibility: The small AUC gaps (0.01–0.02) observed in frozen embeddings, which nearly vanish under fine-tuning, demonstrate that translated corpora are viable for training classifiers.
• Scope: While the study focuses on Polish, the authors posit that success with this morphologically complex language suggests the method will generalize to related Slavic languages.

Limitations Acknowledged:

• Labels were inherited from English annotations without re-annotation by native Polish speakers, precluding the measurement of cross-cultural label shift.
• Prompt engineering was conducted without native Polish speaker involvement, potentially missing specific cultural nuances not captured by one-shot examples.
• The pipeline was validated on social media discourse; transfer to news or parliamentary corpora may require re-evaluation.