Efficient Multilingual Dialogue Processing via Translation Pipelines and Distilled Language Models

Technical Summary: Efficient Multilingual Dialogue Processing via Translation Pipelines and Distilled Language Models

Problem Statement
Multilingual dialogue understanding presents significant challenges, particularly for low-resource Indic languages. While Large Language Models (LLMs) have demonstrated impressive capabilities in English, their performance on other languages remains limited, especially when processing conversational data that requires deep context understanding, speaker turn tracking, and discourse structure analysis. Existing multilingual models often struggle with consistency and robustness across diverse Indic languages, and training data quality can be questionable. Furthermore, direct multilingual approaches often fail to match the performance of specialized English-centric models due to the disparity in training corpus size and evaluation benchmarks.

Methodology
Team Kl33n3x developed a three-stage pipeline system for the NLPAI4Health 2025 shared task, designed to generate summaries and answer questions from dialogues in eight Indic languages (Marathi, Kannada, Gujarati, Telugu, Tamil, Bangla, Hindi, Assamese) plus English. The system avoids direct fine-tuning of a multilingual model for the specific tasks, which initially proved insufficient in maintaining factual consistency and handling medical terminology. Instead, the architecture employs a "translate-first" strategy:

1. Forward Translation: Dialogues are translated from the source Indic language to English using prajdabre/rotary-indictrans2-indic-en-dist-200M. This model, based on the IndicTrans2 architecture, utilizes rotary position embeddings (RoPE) and is optimized specifically for Indic-English pairs to ensure high-quality translation.
2. Multitask Generation: The English text is processed by a distilled version of the Qwen3-4B-Instruct model (unsloth/Qwen3-4B-Instruct-2507-unsloth-bnb-4bit). This model has been compressed to 2.55 billion parameters via knowledge distillation and quantized to 4-bit. It handles three distinct tasks—narrative summarization, structured summarization, and question answering—via task-specific prompting. A critical feature of this model is its massive context window (up to ~256k tokens), allowing it to process entire clinical conversations in a single forward pass without truncation, thereby preserving discourse coherence and factual consistency.
3. Reverse Translation: The generated English outputs are translated back to the source Indic language using prajdabre/rotary-indictrans2-en-indic-dist-200M, which mirrors the forward translation architecture.

The system utilizes greedy decoding for deterministic outputs and processes data through a preprocessing pipeline that handles speaker attribution, turn boundaries, and language tokenization.

Key Contributions

• Distilled Model Efficiency: The paper demonstrates that a compact, 2.55B parameter distilled model can achieve highly competitive performance across nine languages, significantly reducing computational requirements (approx. 6GB GPU memory) compared to loading multiple specialized models or larger LLMs.
• Translation-Based Pipeline for Low-Resource Languages: The work validates that leveraging high-quality, specialized translation models combined with powerful English-centric generative models can outperform direct multilingual approaches, particularly when training data quality is variable.
• Context-Aware Processing: By utilizing a model with a 256k token context window, the system avoids the information loss associated with truncation or sliding-window strategies common in medical dialogue processing.
• Unified Architecture: The approach uses a single instruction-tuned model for all tasks (summarization and QA), simplifying deployment and maintenance in resource-constrained clinical settings.

Results
The system achieved strong win rates in the NLPAI4Health 2025 competition, with particularly robust performance in Question Answering (QnA) for Marathi (86.7%), Tamil (86.7%), and Hindi (80.0%).

• Automatic Metrics: The system achieved F1 scores ranging from 0.43 to 0.67 for QnA, 0.81 to 0.92 for narrative summarization, and 0.35 to 0.43 for structured summarization.
• Semantic Similarity: High BERTScore values (0.82–0.92) across all tasks indicate strong semantic similarity between generated and reference outputs, suggesting the system effectively captures dialogue meaning even when exact lexical overlap varies.
• Task Sensitivity: Performance varied by task type; narrative summarization benefited most from the translation approach, while structured summarization (key-value) showed more consistent performance across languages, likely due to the robustness of the key-value format against minor translation artifacts.

Significance and Claims
The paper claims that translation-based approaches remain a viable and effective strategy for multilingual NLP tasks, especially when direct multilingual models are unavailable or when training data quality is questionable. The work highlights that compact models, when combined with high-quality translation pipelines, can achieve performance comparable to larger models while offering significant computational advantages. The authors note that while the approach is limited by the quality of the translation models (errors propagate downstream) and introduces latency, it provides a practical solution for processing low-resource Indic languages without the need for task-specific fine-tuning on noisy data. Future work is suggested to explore direct multilingual approaches as models and datasets improve, and to investigate methods for reducing translation dependency.