🤖 machine learning

Geometric Filtering of LLM-Generated Samples for Few-Shot Text Classification

This paper proposes a geometric filtering framework that improves few-shot text classification by selecting LLM-generated synthetic samples based on their Euclidean distance to real class examples in embedding space, achieving significant performance gains over existing methods like SMOTE across diverse datasets and models.

Original authors: Benjamín Schindler, Gonzalo A. Ruz

Published 2026-08-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Benjamín Schindler, Gonzalo A. Ruz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to understand human feelings, like whether a tweet is angry, happy, or sad. You want the robot to be a genius, but you only have a tiny scrapbook of examples—maybe just ten or fifty notes for each emotion. This is the world of "few-shot learning," a tricky corner of artificial intelligence where the computer has to learn a lot from very little. Usually, when humans teach a child, we don't just show them one picture of a cat; we show them many. But when data is scarce, the robot gets confused and makes mistakes.

To fix this, scientists have tried two main tricks. The first is like a math wizard: they take two existing examples and mathematically blend them together to create a "fake" new one. It's fast, but the result often sounds like gibberish to a human. The second trick uses a super-smart AI (a Large Language Model, or LLM) to write brand-new stories that sound perfect and grammatically correct. But here's the catch: just because the story sounds good doesn't mean it belongs in the right category. The AI might write a sentence that sounds like "anger" but actually belongs in the "sadness" pile, or it might write something so weird it doesn't fit anywhere. If you feed all these new stories to your robot teacher, it might get even more confused. The big question researchers are asking is: How do we keep the good, useful new stories and throw away the confusing ones without losing the magic of the AI?

This paper introduces a clever solution called "Geometric Filtering." Think of the robot's brain as a giant, invisible map where every sentence is a dot. Sentences that mean the same thing (like "I am furious" and "This makes me mad") cluster together in tight groups, while different emotions live in different neighborhoods. When the AI generates new sentences, they land somewhere on this map. Some land right in the middle of the "anger" neighborhood, which is perfect. Others land in the middle of the road between "anger" and "sadness," or even in the "happiness" district by mistake.

The authors propose a simple rule: before letting the robot learn from a new AI-generated sentence, we measure the distance between that new sentence and the real, human-written examples it's supposed to match. If the new sentence is close to the real "anger" cluster, we keep it. If it's far away or in the wrong neighborhood, we toss it. It's like a bouncer at a club who only lets in guests who are standing right next to the VIP group, ignoring the ones wandering around the parking lot.

The researchers tested this idea on 13 different datasets, using 5 different types of robot brains (classifiers) and over 6,700 different test setups. They found that this simple "distance check" worked incredibly well. By filtering out the bad samples, their method improved the robot's performance by 2.61 percentage points compared to the old math-blending method (SMOTE). In fact, in nearly 89% of the tests, this new approach won. They also discovered that the more complex the filtering rule got—adding extra checks like "is it similar in a different way?" or "is it dense?"—the worse it performed. The simplest rule, just measuring the straight-line distance, was the champion.

One of the most surprising findings was that this trick works even better when the robot has almost no data to start with. When the robot only had 10 examples per category, the improvement was huge, jumping from a 67% success rate to over 72%. But as the robot was given more real examples (up to 50), the benefit of the filter shrank, because the robot was already learning well on its own. The paper also showed that this method isn't just for classifying emotions; it works for spotting names and places in text (Named Entity Recognition) too, boosting performance by over 9 percentage points without needing any changes to the filter.

The authors are very sure about these results because they ran thousands of simulations and used strict statistical tests to prove the improvements weren't just luck. They also tested the method with five different AI generators from four different companies, and it worked well for all of them, proving that the "bouncer" works regardless of which AI is writing the guest list. However, they note that this method is most useful when data is scarce; if you already have thousands of examples, the filter doesn't add much value.

In the end, the paper suggests that we don't need complicated, multi-step rules to clean up AI-generated data. Sometimes, the simplest geometric check—just seeing how close a new idea is to the truth—is the most powerful tool we have. It turns out that in the world of AI, keeping things simple and sticking to the basics of "closeness" is often the smartest move of all.

Technical Summary: Geometric Filtering of LLM-Generated Samples for Few-Shot Text Classification

Problem Statement
In low-resource natural language processing, text classification is hindered by a scarcity of labeled data, often limited to 10–50 examples per class. While Large Language Models (LLMs) offer a solution by generating synthetic training data, a critical quality heterogeneity exists: generated samples vary significantly in their placement within the embedding space. Some align with the target class distribution, while others fall into peripheral zones or cross-class regions. Indiscriminate inclusion of these samples can degrade classifier performance. Conversely, classical oversampling methods (e.g., SMOTE) provide geometric control but often produce linguistically incoherent vectors. This paper addresses the gap between the linguistic validity of LLMs and the geometric awareness required for effective classifier training.

Methodology
The authors propose a geometric filtering framework that acts as a post-generation validation step for LLM-synthesized text. The process involves:

Generation: LLMs generate a surplus of candidate samples (3x the desired amount) using in-context learning with real examples.
Embedding: All texts (real and synthetic) are encoded into L2-normalized 768-dimensional vectors using the all-mpnet-base-v2 model.
Geometric Filtering: The framework evaluates synthetic samples based on their Euclidean distance to real class examples. Five strategies were tested:
- No Filter: Random selection (control).
- LOF Filter: Local Outlier Factor based on density (relaxed and strict variants).
- Cascade Filter: A hierarchical scoring system combining Euclidean distance, cosine similarity, KNN purity, and centroid distance.
- Combined Filter: Requiring simultaneous passage of LOF and cosine thresholds.
- Key Finding: The simplest strategy, Level-1 Cascade (Euclidean distance only), consistently outperformed complex multi-criteria alternatives.
Soft Weighting: Instead of binary acceptance/rejection, filter scores are transformed into continuous sample weights. Scores are normalized and scaled via a temperature parameter ( $T=0.5$ ) to sharpen the distribution toward high-quality candidates. These weights are passed to the classifier during training.

Key Contributions

A Decoupled Filtering Framework: A system that evaluates LLM-generated samples based on embedding space geometry, validated across 5 LLMs from 4 providers (including Gemini, GPT, Claude, Kimi, and GLM).
Soft Weighting Mechanism: A method to convert geometric filter scores into continuous training weights, allowing the model to learn from high-quality synthetic data more heavily than low-quality data.
Cross-Task Generalization: Demonstration that the same geometric filter, without modification, improves performance in Named Entity Recognition (NER) tasks, not just text classification.
Empirical Evidence on Simplicity: Comprehensive analysis across 6,700+ configurations proving that the simplest distance-based filter outperforms complex multi-criteria approaches. The paper attributes this to metric correlation (Euclidean distance and cosine similarity are monotonic in L2-normalized spaces), diversity loss in restrictive filters, and the instability of density estimators (LOF) in few-shot regimes.

Experimental Results
The method was evaluated across 13 datasets, 5 classifiers, and 10 augmentation baselines:

Text Classification: The soft-weighted geometric filtering method achieved a +2.61 percentage point (pp) improvement over SMOTE ( $p < 0.0001$ , Cohen's $d = 0.95$ , 88.9% win rate). Binary filtering alone also yielded significant gains (+2.39pp).
Few-Shot Sensitivity: Benefits were most pronounced in extreme low-resource settings. At 10-shot, the improvement over SMOTE was +4.89pp ( $d=1.48$ ). Gains diminished as the number of real examples increased (e.g., +0.76pp at 50-shot).
Classifier Performance: Linear classifiers (Linear SVC, Ridge) benefited most (+2.98pp and +2.74pp respectively), while tree-based methods (Random Forest) showed smaller but significant gains.
NER Extension: Applied to Named Entity Recognition without modification, the method achieved a +9.26pp improvement over the unfiltered baseline with a 100% win rate.
LLM Robustness: Results were consistent across 5 different generative models, with no statistically significant difference in performance between the best and worst models.

Significance and Claims
The paper claims that the primary value of LLM-based augmentation lies not in the volume of generated text, but in the geometric consistency of the samples with the real data distribution. The authors argue that:

Simplicity is Optimal: Complex filtering criteria often degrade performance by eliminating diverse candidates or relying on unstable density estimates in low-data regimes. A simple Euclidean distance check is sufficient and superior.
Targeted Improvement: The method disproportionately benefits difficult or underrepresented classes (those with baseline F1 < 30%), gaining an average of +10.44pp for these classes compared to +0.55pp for easy classes.
Decoupling: The geometric filtering framework is robust and independent of the specific generative model used, making it a generalizable tool for few-shot learning.

The authors conclude that while LLMs provide linguistic validity, geometric filtering is the necessary mechanism to ensure these samples are statistically useful for training classifiers, particularly when real labeled data is scarce.

Technical Summary: Geometric Filtering of LLM-Generated Samples for Few-Shot Text Classification

More like this