💬 NLP

How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?

This paper demonstrates that strategically prompting commercial and open-source LLMs with Ukrainian-specific minimal-edit instructions and few-shot examples can achieve near state-of-the-art performance in Ukrainian grammatical error correction, closing over 90% of the gap to fine-tuned models while revealing specific linguistic overcorrection patterns.

Original authors: Kateryna Karpo, Artem Chernodub

Published 2026-06-09

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kateryna Karpo, Artem Chernodub

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a friend who is learning to write perfect Ukrainian. You want to help them fix their mistakes, but you have a very specific rule: you can only fix the actual errors. You cannot rewrite their sentences to sound "fancier," change their word choices, or improve their style. You just want to fix the typos, the missing commas, and the wrong grammar, leaving their unique voice exactly as it is.

This paper is about teaching powerful AI "brains" (Large Language Models) to follow that strict rule without needing to be retrained from scratch.

Here is the story of their experiment, explained simply:

The Problem: The Over-enthusiastic Editor

The researchers tried using famous AI models (like those from Google, OpenAI, and Anthropic) to fix Ukrainian text.

The Issue: When they just asked the AI, "Fix this sentence," the AI acted like an over-enthusiastic editor. It didn't just fix the typos; it rewrote the whole sentence, swapped words for synonyms, and changed the style.
The Result: Because the AI changed things that weren't actually broken, it got a low score. It was "correcting" things that didn't need fixing.

The Solution: The "Minimal-Edit" Rulebook

The researchers realized that to get the AI to behave, they needed to give it a very specific, detailed instruction manual written in Ukrainian. They called this the "Minimal-Edits" strategy.

Think of it like giving a robot a strict recipe:

The Rule: "Only fix obvious mistakes like spelling and punctuation."
The Ban: "Do NOT change the author's words. Do NOT use synonyms. Do NOT rewrite the sentence."
The Specifics: They included tiny, language-specific rules, like "Use a long dash (—) for dialogue, not a short hyphen (-)" or "Change the name 'Nastya' to 'Naste' when talking directly to her."

The Experiment: Testing the Rules

They tested 11 different commercial AI models and one open-source Ukrainian model. They tried four different ways of asking the AI to work:

Zero-shot: Just a simple command ("Fix this").
Few-shot: A simple command plus a few examples of correct fixes.
Minimal-Edits (Zero-shot): The strict rulebook, but no examples.
Minimal-Edits (Few-shot): The strict rulebook plus examples.

The Big Discovery:
The "Minimal-Edits" instructions were the magic key. Even without examples, telling the AI exactly what not to do (in Ukrainian) stopped it from rewriting the text. When they combined the strict rules with a few examples, the AI became incredibly accurate.

The "Coach" (AI-Assisted Optimization)

The researchers didn't stop there. They used one AI to act as a "coach" for the prompts.

The coach looked at where the AI made mistakes.
It then tweaked the instruction manual to say, "Hey, stop doing this specific thing."
It did this over and over until the instructions were perfect.

The Results: Closing the Gap

Before this study, the best AI could only get about 27 points out of 100 on this specific test (using an older model). The best human-tuned AI (which requires expensive computer hardware) got 73 points.

By using these clever prompts and the "coach" method, the researchers got a commercial AI (Gemini 3.1-Pro) to score 69 points.

What this means: They closed 90% of the gap between a cheap, easy-to-use AI and the expensive, super-computer-trained AI, just by changing how they asked the question.

The Catch: The "Too-Cautious" AI

There was one downside. The strict rules worked so well that the AI became too careful.

It stopped making mistakes on punctuation and grammar (great!).
But it also stopped fixing some rare, tricky grammar errors because it was afraid to touch them (bad!).
It's like a guard who stops letting in bad people, but accidentally stops letting in some good people too because they look suspicious.

Summary

The paper shows that you don't always need to build a custom, expensive AI to fix Ukrainian grammar perfectly. If you give a standard, powerful AI a very detailed, language-specific set of rules (written in Ukrainian) and a few examples, it can perform almost as well as the best specialized systems. The secret isn't a bigger brain; it's a better instruction manual.

Technical Summary: Prompting for Minimal-Edit Ukrainian Grammatical Error Correction

Problem Statement
Grammatical Error Correction (GEC) for Ukrainian has traditionally relied on fine-tuned Large Language Models (LLMs), which require dedicated GPU infrastructure. While fine-tuned models currently dominate benchmarks, commercial API-accessed LLMs remain largely untested for minimal-edit GEC—a paradigm where the system must correct only clear grammatical, spelling, and punctuation errors while preserving the author's original wording and voice. Previous attempts using API-accessed models (e.g., GPT-3.5 in a zero-shot setting) yielded poor results ( $F_{0.5} = 27.4$ ), far below the fine-tuned State-of-the-Art (SOTA) of $F_{0.5} = 73.14$ . The central question is whether advanced prompting strategies can close this gap without the computational cost of fine-tuning.

Methodology
The authors evaluated 11 commercial LLMs from four providers (OpenAI, Anthropic, Google, Moonshot) and one open-source Ukrainian model (Lapa v0.1.2) on the UNLP 2023 GEC-only benchmark. The study systematically compared four prompting strategies:

Zero-shot: General instructions to correct errors.
Few-shot: Zero-shot instructions augmented with source-target correction pairs from the training set.
Minimal-edits + Zero-shot: Detailed system prompts explicitly defining the scope of corrections (e.g., specific Ukrainian morphological rules, punctuation conventions) and forbidding stylistic changes or synonym substitution.
Minimal-edits + Few-shot: Combining detailed constraints with few-shot examples.

Additionally, the authors employed an LLM-assisted prompt optimization pipeline (using Claude Code skills) to iteratively refine the best manual prompts based on error analysis of the development set. The evaluation used the official UNLP 2023 pipeline, computing span-level Precision, Recall, and $F_{0.5}$ using a Ukrainian adaptation of ERRANT.

Key Findings and Results

Performance Gap: The best zero-shot configuration (Claude Opus 4.6 with Ukrainian instructions) reached $F_{0.5} = 49.70$ , nearly doubling the previous GPT-3.5 baseline but still lagging significantly behind fine-tuned SOTA.
Prompt Language Sensitivity: For zero-shot and few-shot prompts, English instructions generally performed better or equal to Ukrainian instructions for most models. Claude models were the exception, showing performance gains when prompted in Ukrainian. However, for the minimal-edits strategies, Ukrainian instructions were essential because the specific linguistic rules (e.g., vocative case paradigms, euphonic preposition alternations) could not be precisely expressed in English.
Impact of Constraints: The minimal-edits constraint was the most significant factor, outperforming both zero-shot and few-shot baselines across all models. It drastically reduced overcorrection (false positives) by preventing models from rewriting sentences or substituting synonyms.
LLM-Assisted Optimization: Applying LLM-assisted optimization to the best manual prompt (Minimal-edits + Few-shot) yielded further improvements. The best overall configuration, Gemini 3.1-Pro with optimized prompts, achieved $F_{0.5} = 69.22$ . This result closes over 90% of the performance gap between the previous API-accessed baseline and the fine-tuned SOTA ($73.14$).
Error Analysis: Detailed minimal-edits instructions significantly improved performance on rule-based categories like punctuation and case errors. However, they caused the model to become overly conservative on low-frequency grammatical categories (e.g., prepositions, verb voice), leading to a drop in recall for those specific types. The study identified five recurring overcorrection patterns specific to Ukrainian, such as the indiscriminate normalization of en-dashes to em-dashes and the collapse of acceptable morphological variants.

Significance and Claims
The paper claims to be the first systematic evaluation of prompting strategies for minimal-edit Ukrainian GEC using API-accessed LLMs. Its primary significance lies in demonstrating that prompting alone can be competitive with fine-tuned systems, reducing the gap from ~45 points to ~4 points on the $F_{0.5}$ metric.

The authors emphasize four key takeaways:

Instruction Language: While English suffices for general prompts, Ukrainian is required for precise minimal-edits constraints.
Strategy Hierarchy: Minimal-edits constraints provide the largest gains, followed by few-shot examples.
Optimization Transferability: LLM-assisted prompt optimization improves performance on the model family it was tuned for (e.g., Gemini) but does not reliably transfer to other families (e.g., Claude models degraded with optimized prompts).
Trade-offs: Detailed prompting improves precision on high-frequency, rule-based errors but risks suppressing corrections on low-frequency, ambiguous grammatical categories.

The work concludes that while fine-tuned models remain the SOTA, carefully engineered prompting strategies offer a lightweight, accessible alternative that achieves near-SOTA performance for Ukrainian GEC.