💻 computer science

LLM-XTM: Enhancing Cross-Lingual Topic Models with Large Language Models

The paper proposes LLM-XTM, a black-box framework that enhances cross-lingual topic models by integrating LLM-guided refinement with self-consistency uncertainty quantification to achieve superior topic coherence and alignment while reducing reliance on bilingual resources and costly LLM calls.

Original authors: Minh Chu Xuan, Tien-Phat Nguyen, Linh Ngo Van, Dinh Viet Sang, Nguyen Thi Ngoc Diep, Trung Le

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minh Chu Xuan, Tien-Phat Nguyen, Linh Ngo Van, Dinh Viet Sang, Nguyen Thi Ngoc Diep, Trung Le

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the editor of a massive library that contains books written in many different languages. Your goal is to organize these books into "shelves" (topics) so that a book about "cooking" in English sits on the same shelf as a book about "cooking" in Chinese, even if the words used are completely different.

This is the problem of Cross-Lingual Topic Modeling. The paper introduces a new system called LLM-XTM to solve this, and here is how it works, explained simply.

The Problem: The "Noisy" Librarian

Traditional computer programs try to group these books by looking at the words. However, they often make mistakes.

The Mix-up: Sometimes, the computer puts a book about "shoes" on the same shelf as a book about "finance" just because they share a few common words.
The Translation Gap: If you ask the computer to find the "cooking" shelf in English and Chinese, it might give you a list of words that look related but actually mean different things.
The Old Fix: Previous attempts to fix this relied on expensive bilingual dictionaries or parallel texts (books that are exact translations of each other). But these resources are often incomplete, like trying to learn a language with a dictionary that only has 10% of the words.

The Solution: The "Smart Editor" (LLM-XTM)

The authors propose LLM-XTM, which acts like a super-smart editor (a Large Language Model) that helps the computer librarian organize the books. Instead of just guessing, this editor uses its deep understanding of language to clean up the lists.

The system works in two main stages, like a two-step editing process:

Step 1: Cleaning the "Word Lists" (The Self-Consistency Check)

Imagine the computer librarian has written down a list of 15 words for a topic, say "Music." The list might be messy, containing words like "song," "lyrics," but also random words like "marry" or "travel" that got mixed in by accident.

The Human Analogy: If you ask a human editor to fix this list, they might make a mistake or get tired. If you ask them once, you get one opinion.
The LLM-XTM Trick: The system asks the "Smart Editor" (the LLM) to fix the list multiple times (e.g., 5 times).
- Round 1: The editor suggests removing "marry."
- Round 2: The editor suggests removing "travel."
- Round 3: The editor suggests removing "marry" again.
The Result: The system only keeps the words that everyone agrees on across all these rounds. If a word appears in all 5 lists, it's a keeper. If it only appeared once, it was likely a mistake (a "hallucination") and gets thrown out. This ensures the final list of words is stable and makes sense.

Step 2: Aligning the "Shelves" (The Question & Answer Game)

Now that the word lists are clean, the system needs to make sure the "Music" shelf in English is exactly the same as the "Music" shelf in Chinese.

The Analogy: Imagine the computer has a "Question" (a document in English) and a "Pool of Answers" (the topics).
The Process: The system treats every document as a question asking, "What is my main theme?" It then uses a powerful translator (a multilingual encoder) to turn the "Music" topic into a universal meaning.
The Match: It checks if the English document and the Chinese document are asking for the same "answer." If they are, the system forces them to sit on the same shelf. This ensures that a document about "buying a guitar" in English and a document about "buying a guitar" in Chinese end up with the exact same topic distribution, even if the words are totally different.

Why is this better?

It's Black-Box Friendly: You don't need to see the internal "thoughts" of the AI to use it. You just ask it to refine the list, and it does.
It Reduces Hallucinations: By asking the AI the same question multiple times and only keeping the answers it agrees on, the system avoids making up weird or irrelevant words.
It Needs Less Data: It doesn't need perfect, pre-translated books to work. It can take messy, unaligned data and clean it up using the AI's internal knowledge.

The Results

The authors tested this on real-world data, like news articles and product reviews in English, Chinese, and Japanese.

Before: The computer's topic lists were often confusing, mixing unrelated words (like "shoe" and "finance").
After: The lists became much clearer and more consistent across languages. The "Music" topic in English and Chinese now shared the same clear meaning.
Efficiency: They found that asking the AI to refine the list about 5 times was the "sweet spot"—enough to be accurate, but not so many that it became too slow or expensive.

In Summary

LLM-XTM is like hiring a team of expert editors to review a messy library. Instead of just guessing which books go together, they repeatedly check their work to ensure accuracy, and then use a universal "meaning" system to make sure books in different languages end up on the exact same shelves. The result is a library where the topics are clear, consistent, and truly understood across language barriers.

Technical Summary: LLM-XTM

Problem Statement

Cross-lingual Topic Modeling (CLTM) aims to discover shared semantic structures across languages, mapping semantically equivalent documents to comparable topic distributions. However, existing models face significant limitations:

Resource Dependence: Most approaches rely heavily on sparse bilingual resources (e.g., seed dictionaries, parallel corpora, or document embeddings) which often capture only surface-level correspondences and fail to ensure deep thematic consistency.
Noise and Misalignment: Imperfections in corpora, such as mistranslations and domain mismatches, introduce noise, causing nominally aligned topics to diverge in meaning (as illustrated by the misaligned examples in Table 1 of the paper).
Limitations of Current LLM Approaches: While Large Language Models (LLMs) offer deep semantic understanding, recent attempts to use them for topic modeling suffer from:
- Cost and Scalability: Running LLM inference for every document is computationally prohibitive.
- Hallucination: LLMs can generate irrelevant or misleading topic words.
- White-Box Requirements: Some methods (e.g., LLM-ITL) require inaccessible token probabilities to estimate confidence, making them infeasible for closed-source models or low-resource settings.
- Lack of Global Structure: Treating LLM outputs as ground truth for individual documents often overlooks the global corpus structure, leading to incomplete coverage.

Methodology: LLM-XTM

The authors propose LLM-XTM, a two-phase framework designed to enhance pre-trained cross-lingual topic models (such as VAE-based InfoCTM, NMTM, or XTRA) through a scalable, black-box, and uncertainty-aware design. The enhancement phase consists of two primary components:

1. LLM-Guided Topic Word Refinement with Self-Consistency

Instead of treating LLM outputs as absolute ground truth, the framework employs a self-consistency strategy to quantify uncertainty and filter hallucinations:

Candidate Pooling: The base model generates top- $N$ words for each topic in both languages. These are concatenated into a bilingual candidate pool.
Multi-Pass Refinement: The pool is submitted to an LLM $R$ times. The LLM is prompted to remove noisy words, retain representative terms, and add missing coherent terms.
Self-Consistency Aggregation: Word occurrences across the $R$ refinement rounds are aggregated to estimate empirical frequencies. Only words with high agreement across multiple runs are retained, producing a stable, refined bilingual set $\bar{w}_k$ .
MMD Alignment Loss: To ensure the model's internal topic-word distributions ( $\beta$ ) remain consistent with the corpus-driven signals while adopting the LLM's semantic improvements, the framework minimizes the Maximum Mean Discrepancy (MMD) between the model's raw distribution and the LLM-refined target distribution. This acts as a distributional loss to shift the model toward more coherent targets without losing the underlying data structure.

2. Document-Level Alignment via QA Mechanism

To address the issue where semantically equivalent documents in different languages yield diverging topic proportions ( $\theta_d$ ) due to lexical divergence, the authors introduce a novel alignment mechanism inspired by Question Answering (QA):

Semantic Mapping: Documents are treated as "questions" ("What is my core semantic theme?"), and the LLM-refined topics act as "candidate answers."
Embedding Space: Both documents and refined topic word sets are encoded into a shared multilingual semantic space using a powerful encoder (e.g., BGE-M3).
Similarity-Based Target: The similarity between a document embedding and topic embeddings is calculated to derive a target topic distribution $\hat{\theta}_d$ that reflects pure semantic relevance.
KL-Divergence Loss: The model minimizes the KL-divergence $KL(\theta_d \parallel \hat{\theta}_d)$ , forcing the latent topic proportions to align with the semantic intent derived from the LLM-refined topics, thereby enforcing robust cross-lingual consistency at the document level.

The overall objective function combines the backbone's original loss with the MMD refinement loss and the document alignment loss.

Key Contributions

Framework Innovation: Introduction of LLM-XTM, the first framework to integrate LLM-guided refinement into cross-lingual topic modeling while maintaining efficiency and scalability through a black-box approach.
Uncertainty-Aware Refinement: Adaptation of self-consistency techniques (inspired by Self-CheckGPT) to the topic refinement setting. This enables the integration of LLM feedback while filtering out hallucinations, without requiring white-box access to token probabilities.
Novel Alignment Mechanism: A document-level alignment strategy that treats topics as "answers" and documents as "questions," utilizing a KL-divergence objective to align latent distributions with semantic similarity vectors in a multilingual space.
Empirical Validation: Extensive experiments demonstrating that LLM-XTM improves topic coherence and cross-lingual alignment across diverse backbones and datasets.

Experimental Results

The framework was evaluated on three public benchmarks: EC News (English-Chinese), Amazon Review (English-Chinese), and Rakuten Amazon (Japanese-English).

Topic Quality: LLM-XTM consistently improved Topic Coherence (CNPMI) and Topic Quality (TQ) across all baselines (MCTA, MTAnchor, u-SVD, SVD-LR, XTRA, InfoCTM, NMTM).
- For example, on the EC News dataset, integrating LLM-XTM with InfoCTM increased CNPMI by 51.2% and TQ by 43.6%.
- On the Amazon Review dataset, coherence improved by 35.8% for XTRA and 35.1% for InfoCTM.
Cross-Lingual Alignment: The method significantly enhanced cross-lingual transfer performance in classification tasks. On Rakuten Amazon, cross-lingual accuracy for InfoCTM improved from 0.691 to 0.731 (+5.8%), and for NMTM from 0.610 to 0.621 (+1.8%).
Ablation Studies: Removing the document alignment loss ( $L_{doc-align}$ ) notably weakened cross-lingual alignment, while removing the MMD loss ( $L_{MMD}$ ) impaired topic-word coherence. The self-consistency mechanism was shown to be critical for maintaining high coherence.
Qualitative Analysis: Visual inspection of topic words (Table 4) confirmed that LLM-XTM successfully removed noisy, off-topic terms (e.g., "marry" in a music topic) and strengthened semantic alignment (e.g., aligning "skin/fragrance" with "moisturizing/face wash").
Generalization: The framework demonstrated effectiveness on long-document datasets (Airiti Thesis) and was shown to be model-agnostic, performing well with various open-weight LLMs (Llama-3.3, Qwen3, Mistral) in addition to proprietary APIs.

Significance and Claims

The paper claims that LLM-XTM provides a plug-and-play enhancement for existing cross-lingual topic models. Its significance lies in:

Reducing Reliance on Bilingual Resources: By leveraging the internal semantic knowledge of LLMs, it reduces the need for expensive bilingual dictionaries or parallel corpora.
Balancing Stability and Cost: Through self-consistency and periodic refinement (rather than per-document inference), it offers a stable, hallucination-resistant approach that is computationally feasible at scale.
Improving Interpretability: The framework produces topics that are not only statistically coherent but also semantically aligned across languages, addressing the "incoherent or weakly aligned" issues prevalent in current CLTM methods.

The authors acknowledge that the framework's performance is tethered to the quality of the underlying backbone model and that the reliance on external LLM API calls introduces latency, making it less suitable for real-time, highly resource-constrained environments. However, they position LLM-XTM as a robust step forward in making cross-lingual semantic understanding more accessible and reliable.