PSK at WMT 2026 MIST: Task-Specialized QLoRA Adapters for Multilingual Summarization and Question Answering

In the rapidly evolving field of artificial intelligence, researchers are constantly trying to teach computers to understand and generate human language across the globe. A central challenge in this work is making models that can handle many different tasks, such as summarizing a long article or answering a specific question, without needing a separate, massive computer brain for every single job. The goal is to create systems that are efficient enough to run on standard hardware yet flexible enough to switch between languages like Hindi, English, or Yoruba and between tasks like writing a summary or solving a riddle. This balance is crucial because while powerful language models exist, they are often too large to be practical for everyday use or for specific, localized needs. The question driving recent work is whether a single, smaller model can be taught to do many things well by attaching small, specialized tools to it, rather than building a giant new machine for every new problem.

A team of independent researchers recently tackled this challenge by entering a major international competition designed to test how well machines can follow instructions in dozens of languages. Their approach relied on a specific strategy: taking one core language model and attaching three distinct, lightweight add-ons to it. Imagine the core model as a versatile, multilingual engine that knows the grammar and vocabulary of seventy different languages. On its own, this engine is good at general conversation, but it struggles when asked to perform specific, structured jobs like writing a scientific abstract or finding an answer hidden inside a long text. To fix this, the researchers built three separate "adapters," which are essentially small, specialized training modules. One adapter learned how to summarize documents, another learned how to answer questions based on a provided text, and the third learned how to answer open-ended questions from general knowledge.

The researchers started with a model called Tiny Aya Global, which contains roughly 3.35 billion parameters, a measure of its complexity and capacity. This size was chosen specifically because it fits within the strict limits of the competition, which required systems to stay under a 10-billion-parameter threshold. Instead of retraining the entire massive model from scratch, which would be incredibly slow and expensive, they used a technique called QLoRA. This method allows them to freeze the main model and only train the tiny, specialized adapters. They prepared these adapters using different sets of data. For the summarization tool, they fed it a mix of general news articles and scientific papers, pairing full texts with the abstracts written by the author. For the question-answering tools, they used a variety of datasets containing questions and answers in many languages, including some that required the model to connect information across different sentences.

When the team tested their system, they found that separating the tasks worked significantly better than trying to teach the model to do everything at once. They initially tried a single "multitask" adapter trained on a mix of all the data, but this approach produced weaker results. In contrast, the system with three separate adapters showed clear improvements. The summarization adapter, trained on both general and scientific texts, produced summaries that were much closer to the human-written examples than the multitask version. Similarly, the question-answering adapter trained specifically on context-based questions performed better at finding the right answers within a text. The researchers noted that the open-ended question-answering task was more unpredictable; while one version of the adapter performed better on computer-generated scores, another version was better at handling longer, more complex questions without running out of space or repeating itself.

To ensure their system was robust, the researchers tested it on a held-out set of examples that the model had never seen before. They measured success using several standard methods, including how closely the computer's output matched the human answer word-for-word and how similar the overall meaning was. The results showed that their specialized approach improved performance in the majority of the twenty-four languages tested. For summarization, the system improved its output quality across all languages. For context-based questions, it improved in nineteen of the twenty-five languages tested. The open-ended questions remained the most difficult area, with results varying depending on the length of the question and the method used to judge the answer. Because of this uncertainty, the team decided to submit three different versions of their system to the competition, all using the same strong summarization and context-answering tools, but each using a different strategy for the open-ended questions.

The study highlights a practical path forward for building efficient, multilingual AI. By keeping a single, shared foundation and adding small, task-specific tools, the researchers were able to create a system that outperformed a one-size-fits-all approach without requiring massive computational resources. They demonstrated that for tasks like summarizing scientific papers or answering questions based on a text, specialization is key. However, they also acknowledged that open-ended generation remains a complex challenge where automatic scoring methods do not always align with human judgment. The work suggests that while we may not yet have a perfect universal answer generator, we can build highly effective, specialized tools that work well within their specific domains, offering a more reliable and efficient way to interact with technology across the world's many languages.