🤖 machine learning

Annotation Entropy Predicts Per-Example Learning Dynamics in LoRA Fine-Tuning

This paper demonstrates that LoRA fine-tuning uniquely causes model performance to degrade on examples with high annotator disagreement (high annotation entropy), a phenomenon characterized by increasing loss during training that is strongly correlated with annotation uncertainty and distinct from full fine-tuning.

Original authors: Brady Steele

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Brady Steele

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a class of students (the AI models) to solve a tricky logic puzzle called "Natural Language Inference." The puzzles come from a textbook where, for every single question, 100 different people tried to answer it.

Sometimes, all 100 people agree on the answer. We call these "Clean" examples.
Sometimes, the 100 people are split down the middle, arguing back and forth. We call these "Contested" examples.

The paper investigates what happens when you teach these students using a specific, highly efficient method called LoRA (Low-Rank Adaptation). LoRA is like giving the students a very small, limited notebook to write their new notes in, rather than letting them rewrite their entire textbook.

Here is the surprising discovery the authors made, explained simply:

1. The "Un-Learning" Phenomenon

When you teach with a full textbook (Full Fine-Tuning), the students get better at everything over time. Even the hard, controversial questions eventually get easier.

But when you use the LoRA method (the small notebook), something strange happens:

The Easy Stuff: The students get great at the "Clean" examples where everyone agreed.
The Hard Stuff: For the "Contested" examples (where humans disagreed), the students actually get worse as the training goes on. Their mistakes increase.

The authors call this "Un-learning." It's as if the student, in their rush to master the easy, clear-cut facts, accidentally forgets how to handle the messy, ambiguous ones. They specialize so much on the "easy" patterns that they start actively failing the "hard" ones.

2. The "Confusion Meter" (Annotation Entropy)

How did they know which examples were hard? They used a "Confusion Meter" called Annotation Entropy.

Low Entropy: Everyone agrees. The meter is calm.
High Entropy: Everyone is arguing. The meter is chaotic.

The paper found a direct link: The more confused the humans were about an example, the more the AI model "un-learned" it. The model's performance on these confusing examples actually got worse the longer it trained.

3. Why Does This Happen? (The Analogy)

Think of the AI model as a chef trying to learn a new recipe using a tiny, low-quality notepad (LoRA).

The Strategy: The chef sees that 80% of the time, people want a simple salad (Clean examples). They write a perfect, crisp recipe for the salad in their tiny notepad.
The Problem: But 20% of the time, people are arguing about a complex, spicy stew (Contested examples). Because the notepad is so small and the chef is so focused on the salad, they start overwriting the notes about the stew to make room for more salad details.
The Result: By the end of the week, the chef makes the best salad in the world but has completely forgotten how to make the stew, or worse, they start making the stew worse than they did on day one.

In contrast, if the chef had a giant whiteboard (Full Fine-Tuning), they could write down the salad recipe and the stew recipe side-by-side without erasing one to write the other.

4. The "Decoder" vs. "Encoder" Difference

The researchers tested different types of AI "brains."

Encoders (like BERT): These are like students who are good at reading and understanding text. They showed the "un-learning" effect, but it was a bit milder.
Decoders (like Qwen): These are like students who are great at generating text. They showed the "un-learning" effect much more strongly. When these models used the tiny notepad, they forgot the controversial examples even faster than the others.

5. Why Should We Care?

This is a big deal for anyone using AI today.

The Trap: We often use LoRA because it's cheap and fast. But this paper warns us: If your data is messy or controversial, LoRA might make your model worse at handling that mess. It might make the model confident but wrong on the tricky stuff.
The Fix: If you are dealing with data where humans disagree (like medical diagnoses, toxic content detection, or sensitive social issues), you might need to use a bigger "notebook" (higher LoRA rank) or a different training method entirely to make sure the model doesn't "un-learn" the hard truths.

Summary

In short: When you squeeze an AI into a small, efficient training mode (LoRA), it becomes a specialist that ignores the messy, controversial parts of reality. It gets so good at the easy, agreed-upon facts that it actively starts failing the hard, ambiguous ones.

The paper gives us a new way to predict exactly which examples an AI will struggle with just by looking at how much humans disagree on them.

1. Problem Statement

Neural networks typically learn training examples in a consistent order, acquiring "easy" examples before "hard" ones. Existing methods for analyzing training dynamics (e.g., Dataset Cartography) rely on endogenous measures derived from the model's own internal statistics (confidence, variability) to define difficulty. However, these metrics are circular, as they are derived from the very training run they seek to explain.

Furthermore, while Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA (Low-Rank Adaptation) are widely adopted, the specific per-example learning trajectories under LoRA's rank constraints remain unexplored. The authors ask:

Can an external measure of example ambiguity (annotation entropy) predict learning order?
Does the learning dynamic change qualitatively under LoRA compared to full fine-tuning, particularly for examples with high annotator disagreement?

2. Methodology

Data and Annotation Entropy

Datasets: The study uses subsets of ChaosNLI (derived from SNLI and MNLI), which provides 100 independent annotations per example.
Metric: The authors compute Annotation Entropy ( $H_i$ ) for each example based on the distribution of the 100 labels.
- $H_i = -\sum p_{i,c} \log p_{i,c}$ , where $p_{i,c}$ is the fraction of annotators choosing class $c$ .
- Examples are categorized into three groups: Clean (low entropy, strong consensus), Ambiguous (moderate disagreement), and Contested (high entropy, near-uniform or split distributions).
Ground Truth: Unlike prior work that uses model confidence, this approach uses the external human annotation distribution as a ground truth for ambiguity.

Experimental Setup

Models: Six models were tested:
- Encoders: RoBERTa-base, BERT-base, DistilBERT, DeBERTa v3-base.
- Decoders: Qwen2.5-1.5B, Qwen2.5-3B.
Training Configurations:
- LoRA: Applied with ranks $r \in \{4, 16\}$ (and a sweep up to $r=32$ for some).
- Full Fine-Tuning (Full FT): Used as a capacity upper bound.
- Alternative PEFT: IA3 adapters were tested for comparison.
Learning Dynamics Metric: Instead of discrete "forgetting events," the authors use the Area Under the Loss Curve (AULC) and the Loss Change ( $\Delta \ell$ ) from the start to the end of training.
- Lower AULC indicates faster learning.
- Positive $\Delta \ell$ indicates un-learning (loss increases over time).

Analysis

The core analysis correlates Annotation Entropy ( $H$ ) with AULC using Spearman's rank correlation ( $\rho$ ). They also perform partial correlation controls (controlling for sentence length and gold-label identity) and noise-injection experiments.

3. Key Contributions

Discovery of "Un-Learning" in LoRA: The paper identifies a qualitative pattern where contested examples (high annotation entropy) exhibit increasing loss during LoRA training. This "un-learning" phenomenon is largely absent in Full Fine-Tuning and IA3.
External Predictive Power: Annotation entropy is a robust predictor of per-example learning dynamics across six different model architectures and two datasets. The correlation is positive in all 25 tested conditions.
Rank and Architecture Scaling:
- The correlation between entropy and learning difficulty scales monotonically with LoRA rank ( $r=4 < r=16 < \text{Full FT}$ ).
- Decoder-only models show stronger correlations ( $\rho \approx 0.35\text{--}0.42$ ) than encoder models at matched ranks.
Mechanistic Insight: The un-learning is driven by the low-rank subspace constraint, not gradient direction competition. Despite high gradient alignment between clean and contested examples, the low-rank adapter cannot simultaneously satisfy the loss landscapes of both groups, forcing a trade-off that benefits high-agreement patterns at the expense of contested ones.

4. Key Results

Correlation Strength: The Spearman correlation ( $\rho$ $ρ$ ) between Annotation Entropy and AULC is positive across all conditions ( $0.06 \le \rho \le 0.43$ $0.06 \leq ρ \leq 0.43$ ).
- Decoders: Stronger correlations (e.g., Qwen 3B LoRA $r=16$ : $\rho = 0.422$ ).
- Encoders: Significant but slightly lower (e.g., RoBERTa LoRA $r=4$ : $\rho = 0.308$ ).
- Full FT: Correlations remain positive but the qualitative "un-learning" pattern disappears; loss decreases for all categories.
Un-Learning Quantification:
- Under LoRA, contested examples show a consistent positive $\Delta \ell$ (loss increase). For RoBERTa ( $r=4$ ), contested loss increased by $+0.170$ , while clean examples decreased by $-0.586$.
- Under Full FT, both categories show loss decrease.
- Under IA3, contested examples showed near-zero loss change ( $\Delta \ell \approx -0.003$ ), indicating no un-learning.
Robustness:
- The effect replicates across random seeds and datasets (SNLI and MNLI, though weaker on MNLI due to genre diversity).
- Partial correlation controls confirm the relationship is not driven by sentence length or class imbalance.
- Noise Injection: Replacing clean labels with random noise increased the AULC of clean examples, suggesting label disagreement structure directly shapes learning dynamics.
Gradient Analysis: Counter-intuitively, gradients for clean and contested examples remain highly aligned ( $\cos \theta \approx 0.80$ ) under LoRA, whereas alignment drops significantly under Full FT. This suggests the bottleneck is the capacity of the low-rank subspace to represent conflicting loss landscapes, not a conflict in gradient direction.

5. Significance and Implications

Theoretical Insight: The paper challenges the assumption that low-rank adaptation simply "learns less and forgets less" (as suggested by prior work). Instead, it reveals that LoRA can actively un-learn ambiguous data as it specializes in high-confidence patterns.
Diagnostic Tool: Annotation entropy serves as a model-independent, external ground truth for diagnosing training dynamics, complementing internal model statistics.
Practical Guidance:
- Rank Selection: For tasks with high inherent ambiguity (e.g., medical diagnosis, toxicity detection), low-rank LoRA ( $r=4$ ) may be counterproductive. Higher ranks or full fine-tuning may be necessary to preserve learning on contested examples.
- Loss Functions: Standard cross-entropy with majority-vote labels may exacerbate un-learning on ambiguous data. The authors suggest exploring loss functions that account for label uncertainty (e.g., soft labels).
- Monitoring: Tracking per-example loss trajectories can serve as a real-time diagnostic for detecting un-learning in PEFT scenarios.

In summary, this work demonstrates that the rank constraint in LoRA creates a competitive dynamic where the model prioritizes "clean" examples, actively degrading performance on "contested" examples, a phenomenon that is predictable via external annotation entropy and distinct from standard full fine-tuning behavior.