💬 NLP

Bridging the Reasoning Gap in Vietnamese with Small Language Models via Test-Time Scaling

This paper demonstrates that for Vietnamese Small Language Models, Supervised Fine-Tuning combined with simplified Chain-of-Thought prompting and Self-Consistency outperforms complex agentic workflows in bridging the reasoning gap and achieving pedagogical coherence on edge devices.

Original authors: Bui The Trung, Do Minh Duc, Nguyen Van Vinh, Bui Nguyen Quoc Trinh

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bui The Trung, Do Minh Duc, Nguyen Van Vinh, Bui Nguyen Quoc Trinh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

🧠 The Big Idea: Teaching a Tiny Brain to Think Like a Human

Imagine you have a tiny, super-smart robot (a "Small Language Model" or SLM) that fits in your pocket. This robot, named Qwen3-1.7B, is incredibly knowledgeable. It has read almost the entire internet, so it knows the answers to math problems instantly.

The Problem:
Even though the robot knows the answer, it's bad at explaining it.

If you ask it, "What is 12 times 5?", it might just shout "60!" without showing its work.
If you ask it to solve a tricky word problem, it might get the right number but write the explanation in a confusing way, or use English terms when you asked for Vietnamese.
It's like a genius student who can solve a math problem in their head in 2 seconds but fails the test because they didn't write down the steps the teacher asked for.

This paper is about how the researchers taught this tiny robot to slow down, think step-by-step, and explain itself clearly in Vietnamese, without needing a super-computer to do it.

🛠️ The Three-Step Recipe for Success

The researchers used three main tricks to upgrade this tiny robot:

1. The "Translator" Pipeline (Creating Vi-S1K)

They didn't just translate math problems from English to Vietnamese. That would be like translating a joke word-for-word; it loses the humor.

The Analogy: Imagine you have a recipe for a cake written in French. If you just translate the words, you might end up with "baking powder" instead of "baking soda" because the translator didn't understand the context.
What they did: They used a smart AI (Gemini) to act as a cultural chef. It took hard math problems, translated them into natural-sounding Vietnamese, and made sure the math terms (like "decimal point" or "remainder") matched exactly what Vietnamese school textbooks use. This created a special training book called Vi-S1K.

2. The "Tutor" Training (Supervised Fine-Tuning)

They took the tiny robot and taught it using this new book.

The Analogy: Before training, the robot was like a calculator. It just gave you the number. After training, it became a patient tutor.
The Result: The robot learned that to get a good grade, it can't just say "60." It has to say: "First, I multiply 10 by 5 to get 50. Then I multiply 2 by 5 to get 10. Finally, I add them together to get 60."
The Magic: This training unlocked the robot's ability to "talk" like a human teacher. Its ability to explain things improved by 77%.

3. The "Thinking Time" Strategy (Test-Time Scaling)

The researchers tested different ways to ask the robot to solve problems.

The "ReAct" Trap: They tried a method called ReAct, which forces the robot to say "I am thinking," then "I am acting," then "I observed."
- The Analogy: Imagine asking a tiny child to solve a puzzle, but you force them to stop after every move and say, "I am moving the red block," "I am looking at the blue block." The child gets so tired from talking about what they are doing that they forget how to solve the puzzle.
- The Finding: For this tiny robot, ReAct was too complicated. It wasted the robot's brainpower on formatting instead of math.
The Winner (CoT + Self-Consistency): The best method was Chain-of-Thought (just "think step-by-step") combined with Self-Consistency.
- The Analogy: Instead of asking the robot once, they asked it five times to solve the same problem. Then, they looked at the five answers and picked the one that appeared most often (like a majority vote).
- Why it works: Even if the robot makes a silly mistake once, doing it five times usually cancels out the errors. This is the most reliable way to get a correct answer.

📊 What Did They Learn? (The Takeaways)

Small Models Are Smarter Than We Think: The tiny robot already knew the math (it got 4 out of 5 on accuracy right away). It just didn't know how to talk about it.
Training is the Key: You don't need a giant super-computer to make a small model smart. You just need to teach it the right style of thinking (using the Vi-S1K dataset).
Keep It Simple: For tiny robots (under 2 billion "brain cells"), complex frameworks like ReAct are too heavy. They work best when you let them think naturally (Chain-of-Thought) and double-check their work (Self-Consistency).
The "Tutor" Effect: The biggest improvement wasn't in getting the right number; it was in the explanation. The robot went from a silent calculator to a helpful teacher who can explain why the answer is correct.

🚀 Why Does This Matter?

This research is a big deal because it shows we can put smart, reasoning AI directly onto phones, tablets, and cheap laptops (Edge Devices).

Instead of needing a massive server farm in the cloud to solve a math problem for a student in a rural village, we can now run a "tutor" right on their phone. It's fast, private, and works even without an internet connection.

In short: They took a smart but shy robot, taught it the local language and manners, and showed that if you let it think a little longer and check its work, it can be a fantastic teacher.

1. Problem Statement

The paper addresses the "reasoning gap" in Small Language Models (SLMs), specifically those with fewer than 2 billion parameters (e.g., Qwen3-1.7B). While these models often possess high fluency and latent knowledge, they struggle with:

Coherent Chains of Thought: They frequently fail to maintain logical consistency over multi-step deduction tasks.
Non-English Contexts: The gap is exacerbated in low-to-medium resource languages like Vietnamese, where models struggle with specific terminologies, cultural nuances, and pedagogical structures.
Format vs. Logic: Base models often calculate the correct answer but fail to present the solution in a structured, educational format required for real-world applications (e.g., tutoring).
Inefficiency of Complex Frameworks: It is unclear whether complex agentic frameworks (like ReAct) are viable for sub-2B models or if they impose a "cognitive tax" that degrades performance.

2. Methodology

The study employs a three-pronged approach: Dataset Construction, Supervised Fine-Tuning (SFT), and Test-Time Scaling via various prompting strategies.

A. Model and Architecture

Base Model: Qwen3-1.7B, a dense Transformer with 28 layers and Grouped Query Attention (GQA), pre-trained on ~36 trillion tokens.
Hardware: NVIDIA A100-SXM4-80GB GPU using BF16 precision (no quantization) to preserve weight fidelity.
Inference Engine: vLLM with a context window of 32,768 tokens.

B. Dataset Construction (Vi-S1K & Vi-Elementary-Bench)

Vi-S1K (Training): A high-fidelity dataset of 1,000 math problems derived from the English Simple Scaling S1 dataset.
- Pipeline: Instead of direct translation, the authors used Gemini 2.5 Flash-Lite with context-aware prompts to translate while preserving reasoning traces (CoT).
- Localization: Applied terminology normalization (e.g., converting "decimal point" to Vietnamese "dấu phẩy") and cultural adaptation (currency, names) to align with the Vietnamese Ministry of Education standards.
Vi-Elementary-Bench (Evaluation): A test suite of 1,010 problems stratified into six categories: Logic Puzzles, Typical Word Problems, Explanation/Inference, Math Poems, Math Vocabulary, and Multiple Choice.

C. Training Strategy

Fine-Tuning: Used LoRA (Low-Rank Adaptation) with rank $r=64$ and $\alpha=16$ on the Qwen3-1.7B base model.
Format: ChatML format distinguishing System, User, and Assistant roles to teach the model the "tutor" persona.

D. Prompting Strategies Evaluated

The study compared five strategies to determine the optimal test-time scaling approach:

Zero-shot: Direct generation.
Few-shot: In-context learning with 3 or 5 examples.
Chain-of-Thought (CoT): Explicit "think step-by-step" instruction.
CoT Self-Consistency (CoT-SC): Generating $N$ diverse paths (Temperature 0.7) and aggregating via majority vote ( $N=3, 5$ ).
ReAct: Interleaving Thought, Action, and Observation (adapted for internal cognitive steps).

E. Evaluation Protocol

LLM-as-a-Judge: Used Gemini 2.5 Flash-Lite (Temperature 0.0) to evaluate outputs.
Rubric: A 5-point Likert scale across five dimensions: Accuracy, Completeness, Explanation Quality, Argumentation/Logical Structure, and Cultural/Linguistic Appropriateness. This avoids the brittleness of Exact Match metrics.

3. Key Contributions

Dual-Resource Vietnamese Benchmarks: The release of Vi-S1K (training) and Vi-Elementary-Bench (evaluation), addressing the scarcity of high-quality reasoning data for Vietnamese.
SFT as a "Reasoning Unlocker": Demonstrated that SFT on a small dataset transforms a base model from a "latent calculator" into a "coherent tutor," significantly bridging the gap between raw calculation and pedagogical communication.
Identification of the "Cognitive Tax": Revealed that structured frameworks like ReAct impose a performance penalty on sub-2B models due to attention fragmentation, whereas simplified CoT is more efficient.

4. Key Results

A. Impact of Supervised Fine-Tuning (SFT)

Accuracy: Improved modestly (e.g., CoT accuracy rose from 4.05 to 4.55), indicating the base model already possessed strong latent calculation skills.
Explanation Quality: Improved drastically (+2.00 points on a 5-point scale). The finetuned model successfully learned to articulate steps in Vietnamese, transforming from a calculator to a tutor.
Completeness: Improved by +1.70 points, showing the model learned to include all necessary logical steps.

B. Prompting Strategy Comparison

Best Performance: CoT Self-Consistency (k=5) achieved the highest overall score (4.69), proving that majority voting mitigates the stochastic errors inherent in SLMs.
Best Efficiency: Standard CoT offered the best balance of speed and accuracy (4.59), suitable for real-time interaction.
ReAct Failure: ReAct performed significantly worse (4.11) than CoT, even after fine-tuning. The rigid formatting (Thought/Action/Observation) consumed the model's limited attention capacity, leading to "context crowding" and calculation errors.

C. Quantitative Gains (Post-SFT vs. Pre-SFT)

Metric	Base Model (CoT)	Finetuned Model (CoT)	Improvement
Accuracy	4.05	4.55	+0.50
Explanation Quality	2.60	4.60	+2.00
Completeness	2.80	4.50	+1.70
Overall Score	3.19	4.59	+1.40

5. Significance and Implications

Deployment Hierarchy for Edge AI: The paper establishes a clear hierarchy for deploying SLMs on resource-constrained devices:
1. Optimal: SFT + CoT Self-Consistency (for high-reliability offline grading).
2. Efficient: SFT + Standard CoT (for real-time interaction).
3. Avoid: Complex Agentic workflows (ReAct) for models <2B parameters.
Language Democratization: Proves that "reasoning breakthroughs" observed in English-centric large models can be replicated in Vietnamese SLMs through targeted localization and SFT, rather than requiring massive parameter scaling.
Cost-Effective Evaluation: Validates the use of Gemini 2.5 Flash-Lite as a robust, cost-efficient judge for nuanced mathematical reasoning in low-resource languages, surpassing traditional exact-match metrics.

Conclusion: The study confirms that for Small Language Models, quality of reasoning traces (via SFT) and test-time scaling (via Self-Consistency) are more critical than complex agentic frameworks. This approach enables the deployment of sophisticated, pedagogically sound AI tutors on edge devices using sub-2B parameter models.