🤖 AI

Think in English, Answer in Korean: Efficient Adaptation of Multilingual Tool-Using Agents

This paper introduces LuckyStar 111B, a hybrid reasoning model developed by Cohere and LG CNS that efficiently adapts a post-trained multilingual base model for Korean-English enterprise agents through specific fine-tuning and quantization strategies, significantly enhancing tool-use capabilities like function calling and NL2SQL while maintaining general instruction-following quality under memory constraints.

Original authors: Utsav Garg, Sungjin Hong, Jason Jung, Justin Lee, Shaan Desai, Joon Hee Kim, Anirudh Shrinivason, Edmond Wen, Susie Park

Published 2026-07-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Utsav Garg, Sungjin Hong, Jason Jung, Justin Lee, Shaan Desai, Joon Hee Kim, Anirudh Shrinivason, Edmond Wen, Susie Park

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, multilingual assistant named LuckyStar. This assistant is huge (111 billion "brain cells" or parameters) and works for big companies. The challenge was that companies needed this assistant to be smart enough to solve complex math problems, write computer code, and talk to databases (like SQL), but they also needed it to be fast, cheap to run, and able to speak both Korean and English fluently without getting confused.

Here is the story of how the researchers built and tuned LuckyStar, explained simply:

1. The Starting Point: A Smart Base

Instead of building a new brain from scratch (which is like teaching a baby to read from zero), they started with a very smart, already-trained model called Command A. Think of this as hiring a senior employee who already knows how to follow instructions perfectly in both Korean and English. They just needed to teach them a few new, specific job skills.

2. The "Switch" Trick: Hybrid Reasoning

The biggest innovation was giving the assistant a mental switch controlled by a simple "preamble" (a short instruction at the start of a conversation).

Mode A (The Sprinter): If you ask a simple question like "What's the weather?", the switch tells the model to be quick and concise.
Mode B (The Detective): If you ask a hard math problem or a complex database query, the switch tells the model to "think out loud," taking many steps to solve it before giving the answer.
Why this matters: They didn't need two different robots. One robot could do both jobs perfectly just by changing its "mindset" based on the prompt.

3. The Language Puzzle: Thinking in English, Speaking in Korean

During training, they noticed a weird glitch. When they asked the model a question in Korean, it would often start thinking in Korean but then accidentally finish the answer in English. It was like a student who studies in French but writes their final essay in Spanish.

To fix this, they used a clever strategy:

They taught the model to think (do the complex reasoning) in English because the training data for math and logic was mostly in English.
But they strictly punished the model if it didn't speak the final answer in the user's language (Korean).
The Result: The model learned to do its internal "detective work" in English but always handed the final report in the language the user asked for.

4. The "Tool" Training: Learning to Use Databases

To make the assistant good at "Agentic" tasks (using tools like SQL databases), they had to be very careful.

The Cold Start Problem: At first, the model was terrible at writing database queries. It was like trying to teach someone to drive a race car when they've never held a steering wheel.
The Solution: They used a "try and reject" method. They generated thousands of possible answers, checked which ones actually worked (were "verifiable"), and only fed the correct ones back to the model to learn from. This built a strong foundation before they started the advanced training.

5. The "Chatty" Problem: Learning to Shut Up

After teaching the model to be a great detective, it got a little too chatty. It would give long, rambling explanations even when you just wanted a "Yes" or "No."

The Fix: They used a technique called Preference Alignment. They showed the model examples of "Good, concise answers" vs. "Bad, long-winded answers" and told it, "We prefer the short ones." This tuned the model to be helpful without being annoying.

6. The Hardware Hack: Fitting a Whale in a Bathtub

The model is massive (111 billion parameters). Usually, you need a huge, expensive supercomputer to run it.

The Trick: They used 4-bit quantization. Imagine taking a high-resolution photo and compressing it so much that the file size is half as big, but you can still recognize the picture perfectly.
The Result: This allowed the massive 111-billion-parameter model to run on a single standard graphics card (an H100 GPU) without losing much quality. This makes it much cheaper and easier for companies to actually use.

Summary of Results

The final model, LuckyStar 111B, is a practical, bilingual (Korean/English) agent that:

Solves math and logic problems better than the original base model.
Can talk to databases and use tools effectively.
Speaks the user's language correctly, even when thinking in another.
Can run on a single server, saving money.

The paper concludes that for enterprise use, you don't always need a new, perfect model from scratch. Instead, you can take a smart existing model, give it a "switch" for different tasks, teach it to use tools carefully, and compress it to run efficiently.

Technical Summary: Efficient Adaptation of Multilingual Tool-Using Agents (LuckyStar 111B)

1. Problem Statement

Enterprise assistants operating in multilingual environments face a complex set of constraints: they must reason over private data, execute tool calls (such as SQL or retrieval), and adhere to strict memory and serving budgets. A specific challenge arises in Korean-English bilingual settings where an agent must parse a Korean request, reason over English-heavy schemas or documentation, execute tools, and return a Korean answer with numerically grounded results.

Existing approaches often require training models from scratch or maintaining separate models for different workloads (e.g., reasoning vs. concise interaction), which is costly and inefficient. Furthermore, adapting post-trained multilingual models to verifiable agentic workflows often leads to failure modes such as language drift (Korean prompts resulting in English answers) and verbosity after reasoning optimization. The goal is to adapt a large-scale, post-trained multilingual model to handle hybrid reasoning and tool use efficiently under memory-constrained deployment (e.g., single-GPU serving).

2. Methodology

The authors introduce LuckyStar 111B, a 111B-parameter bilingual model adapted from Cohere's fully post-trained Command A model. Rather than pretraining from scratch, the team employed a three-stage adaptation pipeline to add reasoning and tool-use capabilities while preserving general instruction-following quality.

2.1 Hybrid Adaptation Pipeline

The core design choice is hybrid reasoning via preamble conditioning. The model supports two operating modes using the same weights:

Reasoning Mode: Triggered by a specific preamble, eliciting step-by-step behavior for mathematics, tool use, and Natural Language to SQL (NL2SQL).
Non-Reasoning Mode: Triggered by a different preamble, eliciting concise responses for standard user interactions.

The training pipeline consists of:

Hybrid Supervised Fine-Tuning (SFT): The dataset comprises ~80% reasoning examples (math, code, tool use) and ~20% non-reasoning examples to preserve concise behavior. For reasoning, the model is trained to generate intermediate steps before the final answer.
Reinforcement Learning with Verifiable Rewards (RLVR): This stage optimizes for correctness in multi-step tasks.
- Data Curation: For NL2SQL, the team curated 100,000 verifiable prompt-query pairs from Spider, BIRD, and SynSQL, filtering for executable queries with outputs between 1–500 tokens.
- Cold-Start Solution: Since the base model had <1% accuracy on the curated NL2SQL set, the team used best-of-N rejection sampling with strong internal and open models to generate initial high-quality traces for SFT bootstrapping.
- Reward Functions: Binary rewards are assigned based on rule-based verifiers or an LLM judge (Command A) for correctness.
Offline Preference Alignment (DPO): To counteract the verbosity introduced by RLVR, the team applied Direct Preference Optimization (DPO) with a low learning rate. This stage uses preference pairs where direct answers are "chosen" and verbose outputs are "rejected," refining chatbot behavior without degrading reasoning capabilities.

2.2 Multilingual Reasoning Strategy

Early experiments revealed that Korean prompts often drifted into English reasoning traces and final answers, despite the prompt being in Korean.

Translation Attempt: Translating English reasoning traces to Korean yielded minimal improvement, likely due to degraded naturalness and learning signals.
Mixed-Language Strategy: The team adopted a strategy where the model reasons in English but produces the final answer in the user's language (Korean).
Language Consistency Penalty: During RLVR, a penalty ( $r_{penalty} = 0.5$ ) is applied if the final answer's language mismatches the prompt language, while intermediate reasoning remains in English. This significantly reduced the language mismatch rate from 12.2% (SFT) to 0.8% (final RLVR).

2.3 Efficient Deployment

To address memory constraints, the model was quantized from FP8 to 4-bit weights. This reduced the memory footprint by approximately 50%, enabling the 111B-parameter model to run on a single 80GB H100 GPU.

3. Key Results

The adapted LuckyStar 111B model demonstrates significant improvements in tool use and reasoning while maintaining general multilingual quality.

Reasoning and Tool Use:
- Mathematical Reasoning: Improved Pass@1 accuracy on AIME 2024 (69.3 vs. 10.0 for Command A) and MATH 500 (93.6 vs. 76.2).
- Function Calling: Improved on BFCL v3 (73.7 vs. 13.3).
- Enterprise NL2SQL: Improved from 7.3 to 38.0 on an internal 150-question benchmark.
- LG Agentic Evaluation: Improved from 2.67 to 4.85 on a 60-question business/finance benchmark requiring retrieval and SQL.
- The 4-bit quantized model maintained comparable performance to the FP8 model across these benchmarks.
General Multilingual Quality:
- The model preserved strong instruction-following capabilities. On Korean benchmarks (KMMLU, ARC-C, IFEval, MT-Bench), it achieved scores comparable to or exceeding other strong proprietary models (e.g., GPT-4o, Claude 3.7 Sonnet) and the base Command A model.
- English performance remained robust, with scores on MMLU (82.7) and ARC-C (93.8) matching or exceeding the base model.

4. Significance and Claims

The paper claims to provide a practical recipe for adapting post-trained multilingual models to verifiable agentic workflows under memory-constrained deployment.

Hybrid Reasoning: The use of preamble conditioning allows a single model to switch between concise and reasoning modes, avoiding the need for separate models for different enterprise workloads.
Failure Mode Analysis: The authors identify and address three critical failure modes in scaling agentic systems:
1. NL2SQL Cold-Start: Solved via rejection-sampled SFT to bootstrap RL.
2. Language Drift: Solved via language-consistency rewards during RLVR.
3. Verbosity: Solved via offline preference alignment (DPO).
Deployment Feasibility: The work demonstrates that a 111B-parameter model can be effectively deployed on a single H100 GPU via 4-bit quantization with minimal loss in benchmark quality, making advanced agentic capabilities accessible in enterprise settings with limited hardware.
Separation of Concerns: The results suggest that separating internal reasoning language (English) from user-facing response language (Korean) is a practical, albeit imperfect, strategy for multilingual agents, though native-language reasoning remains a target for future work.

The authors emphasize that while public benchmarks provide comparability, internal enterprise evaluations are crucial for capturing specific workflows, schema constraints, and numerical accuracy requirements necessary for real-world deployment.