An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation

This paper presents a highly efficient, locally deployable Ukrainian RAG system that achieved 2nd place in the UNLP 2026 Shared Task by utilizing a custom two-stage hybrid search pipeline and a fine-tuned, compressed language model optimized for resource-constrained hardware.

The Gist

Imagine you are a librarian in a massive, ancient library filled with thousands of books written in Ukrainian. Suddenly, a researcher rushes in and asks a very specific, tricky question. You don't have time to read every book, you don't have a supercomputer to help you, and you have to give the answer—and prove exactly which page you found it on—within a strict time limit.

This paper describes how a team of researchers built a "Digital Librarian" (an AI system) that can do exactly that, incredibly fast, using only a standard, modest computer.

Here is how their "Digital Librarian" works, broken down into three simple steps:

1. The "Two-Stage Search" (Finding the Right Book)

If you ask a question, you don't want to look through every single word in the library. That would take forever. Instead, the team taught their AI to search in two quick waves:

• Wave 1: The Quick Scan (Document Level). The AI first scans the "titles and summaries" of all the books to find the one or two most likely candidates. It uses two methods: one that looks for the meaning of your question (like a smart person) and one that looks for exact keywords (like a fast scanner).
• Wave 2: The Deep Dive (Page Level). Once it has the right book, it doesn't just flip pages randomly. It breaks the book into small, manageable "snippets" and uses a specialized "refiner" to pick out the exact paragraphs that actually answer your question.

The Analogy: It’s like finding a specific recipe. First, you walk to the "Cooking" section of the library (Document Level). Then, you flip through the index of the cookbook to find the "Dessert" chapter (Page Level).

2. The "Specialized Brain" (Understanding Ukrainian)

Most famous AIs (like ChatGPT) are like polyglots who are "okay" at many languages but are actually "experts" in English. When they try to speak Ukrainian, they sometimes struggle, use too much "mental energy" (memory), or get confused by the grammar.

The researchers didn't want a generalist; they wanted a specialist. They took a model called MamayLM—which was already quite good at Ukrainian—and gave it "extra tutoring." They used a technique called Synthetic Data Generation, where they used a powerful AI to create thousands of practice questions and answers. This was like giving the student a massive stack of practice exams before the real test.

The Analogy: Instead of hiring a general translator who knows a bit of everything, they hired a local Ukrainian professor who has studied these specific textbooks for years.

3. The "Compact Suitcase" (Local Deployment)

The biggest challenge was that this AI had to run on a single, older piece of hardware (a P100 GPU) and finish everything within 9 hours. It couldn't "call home" to a giant cloud server for help; it had to work entirely offline.

To make this possible, they used Quantization. This is a way of shrinking the AI's "brain" without making it lose its intelligence.

The Analogy: Imagine you have a massive, heavy encyclopedia. To make it portable, you don't throw pages away; instead, you use a high-tech "compression" method to shrink the ink so the book becomes as light as a paperback, but you can still read every single word clearly.

The Result

By combining this smart searching, specialized training, and clever shrinking, the team's system won 2nd place in a major international competition (the UNLP 2026 Shared Task). They proved that you don't need a massive, expensive supercomputer to have a highly accurate, "grounded" AI—you just need a very smart, well-organized system.

Technical Summary

Technical Summary: An End-to-End Ukrainian RAG for Local Deployment

Problem Statement

The paper addresses the challenge of building a high-performance Retrieval-Augmented Generation (RAG) system specifically for the Ukrainian language under extreme computational constraints. The primary difficulties identified are:

1. Language Disparity: Most Large Language Models (LLMs) are optimized for English; for Ukrainian, standard tokenizers fragment Cyrillic text into excessive subword pieces, leading to high memory usage, slow inference, and degraded accuracy.
2. Resource Constraints: The system was developed for the UNLP 2026 Shared Task, which required entirely offline inference on a single, legacy NVIDIA P100 GPU within a strict 9-hour time limit.
3. Information Integrity: The need to prevent "hallucinations" by ensuring every answer is grounded in a specific document and page number.

Methodology

The authors propose a modular, three-stage pipeline designed for efficiency and precision:

1. Data Preparation & Hybrid Retrieval

• Layout-Aware Extraction: Uses pymupdf4llm to convert PDFs into Markdown, preserving structural elements like tables and headers.
• Two-Stage Retrieval:
  • Document-Level: A hybrid approach combining Dense Retrieval (using pplx-embed-context-v1-0.6b embeddings) and Sparse Retrieval (BM25). If the top results from both methods do not match, a jina-reranker-v3 is used to select the best document.
  • Page-Level: Documents are split into syntactic chunks (max 500 characters) with a 10% overlap. The system uses Reciprocal Rank Fusion (RRF) to merge dense and sparse rankings, followed by a custom cross-encoder (BAAI/bge-reranker-v2-m3) to rerank the top 8 candidates.

2. Synthetic Data Generation
To overcome the lack of large-scale Ukrainian QA datasets, the authors built an automated pipeline using GPT-4o-mini. This pipeline scans PDF pages and generates approximately 7,000 synthetic multiple-choice questions (MCQs), ensuring each question is self-contained and includes relevant entity names.

3. Model Fine-Tuning & Optimization

• Reranker Fine-Tuning: The BGE reranker was fine-tuned using a hybrid loss objective (Listwise Cross-Entropy and Pointwise BCE).
• Generative Model: The authors fine-tuned MamayLM-12B (a Ukrainian-optimized model based on Gemma 3) using LoRA (Low-Rank Adaptation). The training utilized a "syntactic masking" strategy, where the model was specifically trained to output the correct answer letter and the corresponding page number.
• Deployment Optimization: To fit the 12B model onto the P100 GPU, the model was compressed using 4-bit GGUF quantization via llama.cpp.

Key Contributions

• Specialized Two-Stage Pipeline: A highly efficient document-then-page retrieval architecture.
• Synthetic Dataset Methodology: A scalable way to create high-quality, domain-specific Ukrainian training data.
• Optimized Local Deployment: Demonstrating that a 12B parameter model can be effectively quantized and deployed for high-accuracy, grounded QA on consumer-grade/legacy hardware.

Results

The system achieved 2nd place in the UNLP 2026 Shared Task.

• Performance: Achieved a score of 0.942 on the private test set and 0.920 on the public set.
• Retrieval Accuracy: The two-stage hybrid retrieval achieved near-perfect document identification and a page-level recall@3 of 0.92.
• Efficiency: The entire pipeline successfully processed the hidden test corpus within the mandatory 9-hour window on a single P100 GPU.

Significance

This work proves that high-quality, verifiable AI question-answering is possible for medium-resource languages like Ukrainian without relying on expensive, third-party cloud APIs. By focusing on local deployment and efficient retrieval, the authors provide a blueprint for privacy-preserving, resource-efficient, and factually grounded AI applications in specialized domains (e.g., legal or pharmaceutical) where data security and accuracy are paramount.