🤖 AI

Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation

This paper introduces Synthline, a configurable synthetic data generator that integrates multi-sample prompting and Actor-Critic prompt optimization to overcome the diversity limitations of LLM-generated data, demonstrating that these techniques significantly enhance both data variety and downstream task performance, often surpassing human-authored datasets in data-scarce domains.

Original authors: Abdelkarim El-Hajjami, Camille Salinesi

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abdelkarim El-Hajjami, Camille Salinesi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to understand human language, specifically in the tricky world of software requirements (the instructions people give to build apps and systems). To teach the robot, you need a massive library of examples. But here's the problem: in fields like healthcare or software engineering, real examples are rare, expensive to get, or locked away for privacy reasons.

So, researchers decided to use AI (Large Language Models) to write these examples for them. It's like asking a super-smart ghostwriter to invent thousands of practice test questions.

But there was a catch: The ghostwriter was getting lazy. It kept writing the same boring sentences over and over again. If you teach a robot with boring, repetitive examples, the robot gets confused and fails the real test.

This paper is about two clever tricks the researchers used to fix this lazy ghostwriter and make the AI generate better, more diverse, and more useful practice data.

The Two Magic Tricks

1. The "Batch Cooking" Method (Multi-Sample Prompting)

The Problem: Imagine you ask a chef, "Make me a sandwich." They make one perfect sandwich, but it's the exact same kind of sandwich they make every time. If you ask them to make 20 sandwiches one by one, they will just make 20 identical copies.

The Solution: Instead of asking for one sandwich at a time, the researchers told the AI: "Make me 20 different sandwiches right now, and put them all in a tray."

By forcing the AI to generate a whole batch at once, it has to "think" harder to make sure they aren't all the same. It uses its internal memory of the previous sandwiches it just wrote to avoid repeating itself.

The Result: This was a huge win. The AI produced much more varied data, and when they used this data to train their robot, the robot's performance skyrocketed (improving by up to 44% in some cases). Plus, it was faster and cheaper because they only had to "ask" the AI once instead of 20 times.

2. The "Editor and Critic" Game (PACE)

The Problem: Even with the batch method, the AI sometimes still wrote in a very predictable, robotic style. They needed a way to force the AI to be more creative.

The Solution: They set up a game between two AI characters:

The Actor: Writes a draft of the prompt (the instructions).
The Critic: Reads the draft and says, "This is too boring! Make it sound more different from the others!"
The Loop: The Actor rewrites the instructions based on the Critic's feedback. They do this over and over (like a writer and an editor working together) until the instructions are perfect for generating diverse data.
The Result: This trick was a mixed bag. It definitely made the writing sound more varied (less repetitive words), but it didn't always help the robot learn better.
- For some tasks (like distinguishing "Functional" vs. "Non-Functional" requirements), it helped a lot.
- For others (like spotting security flaws), it actually made the robot worse at the job.
- The Lesson: Just because something sounds "diverse" doesn't mean it's "useful." You can't just optimize for variety; you have to optimize for the specific goal.

The Big Reveal: Can AI Replace Humans?

The researchers asked a scary question: "Can AI-generated practice data be better than real human data?"

The Answer: Yes, sometimes!

When Real Data is Scarce: In tasks where they had very few real examples (like finding defects in software), the AI-generated data was 15% better at training the robot than the tiny pile of real human data. The AI could create a perfectly balanced set of examples, whereas the real data was messy and unbalanced.
When Real Data is Abundant: If they already had a huge library of real human examples, the AI couldn't beat them. Real human data is still the gold standard when you have enough of it.

The Takeaway in Plain English

Don't ask for one thing at a time: If you want an AI to generate training data, ask for a whole batch at once. It forces the AI to be less repetitive and creates better learning material.
Variety isn't everything: Trying to force an AI to be "diverse" using complex editing loops (like the Critic/Actor game) can backfire. It might make the text sound unique, but it might lose the specific details needed to solve the actual problem.
AI is a powerful backup: In fields where real data is hard to get, AI-generated data isn't just a "cheap substitute"—it can actually be superior because it can be perfectly balanced and tailored to the job.

The Bottom Line: The researchers built a tool called Synthline (a "factory" for synthetic data) that uses these tricks. They proved that with the right prompting strategies, we can generate high-quality training data that helps AI learn faster and better, especially in fields where real-world data is hard to find.

1. Problem Statement

High-quality labeled datasets are critical for training machine learning models, but domains like Requirements Engineering (RE) and healthcare face severe data scarcity due to privacy, cost, and proprietary restrictions. While Large Language Models (LLMs) offer a solution via Synthetic Data Generation (SDG), current LLM-generated data suffers from low diversity and high repetition (mode collapse). This is largely attributed to post-training alignment biases that favor stereotypical outputs.

Existing approaches (zero-shot, few-shot, or attribute-based prompting) fail to sufficiently address this diversity gap. Furthermore, there is a lack of empirical understanding regarding whether increasing diversity (e.g., via prompt optimization) actually improves downstream utility (classification performance) or if the two metrics are in conflict.

2. Methodology

The authors propose and evaluate two mechanisms integrated into Synthline, a configurable synthetic data generator based on Feature Models (FM).

A. The Synthline Framework

Synthline allows users to configure generation attributes (domain, style, length, etc.) via a Feature Model. The system was extended to include two specific diversity-enhancing mechanisms:

Multi-Sample Prompting: Instead of generating one sample per prompt, the LLM is instructed to generate multiple samples (e.g., 20) within a single prompt, formatted as a JSON array. This leverages "within-context awareness" to reduce repetition and improves API efficiency (fewer calls).
PACE (Prompt with Actor-Critic Editing): An automated prompt optimization loop.
- Actor: Generates a batch of candidate samples based on the current prompt.
- Critic: Evaluates the batch using a scoring function (maximizing pairwise cosine distance of Sentence-BERT embeddings) and provides natural language feedback.
- Refinement: The Actor uses this feedback to revise the prompt. This cycle repeats for a fixed number of iterations to find a prompt that maximizes lexical diversity.

B. Experimental Setup

Domain: Requirements Engineering (RE).
Tasks: Four classification tasks:
1. Requirements Defects (6 classes).
2. Functional vs. Non-Functional.
3. Quality vs. Non-Quality.
4. Security vs. Non-Security.
Datasets: Real-world datasets (ranging from 131 to 956 samples) used as baselines.
LLM: gpt-4.1-nano-2025-04-14 (used for generation, Actor, and Critic roles).
Variants Tested:
- synthetic_1: Single sample per prompt (baseline).
- synthetic_20: 20 samples per prompt (Multi-sample).
- synthetic_20_pace: 20 samples with PACE optimization.
Evaluation Metrics:
- Diversity: Inter-sample N-gram Frequency (INGF) for lexical diversity; Average Pairwise Similarity (APS) for semantic diversity.
- Utility: Weighted F1-score, Precision, and Recall of BERT classifiers trained on synthetic data and tested on real held-out data.

3. Key Contributions

Empirical Evaluation of Diversity Mechanisms: The first study to rigorously evaluate multi-sample prompting and PACE specifically for SDG in classification tasks.
Synthline Release: A publicly released, Feature Model-based configurable generator that integrates these mechanisms.
Novel Findings on Diversity vs. Utility:
- Demonstrated that Multi-Sample Prompting improves both diversity and utility simultaneously.
- Revealed that PACE improves lexical diversity but has task-dependent effects on utility, proving that optimizing solely for diversity can harm downstream performance.
Synthetic vs. Human Data: Showed that synthetic data can match or surpass human-authored data in scenarios with limited real data, achieving F1-score improvements of up to 15.4 percentage points.

4. Key Results

RQ1: Impact of Multi-Sample Prompting

Diversity: synthetic_20 reduced lexical repetition (INGF) by 25–28% and semantic similarity (APS) by 22–28% compared to synthetic_1.
Utility: Consistently outperformed single-sample generation across all tasks.
- F1-score gains: Ranged from 6 to 43.8 percentage points.
- Best Performance: Security classification saw a massive +43.8 pp gain; Defects saw +11.2 pp.
Efficiency: Reduced API calls by a factor of $1/n$ (where $n$ is samples per prompt).

RQ2: Impact of PACE Optimization

Diversity: PACE consistently improved lexical diversity (lower INGF by 10–19%) across all tasks.
Utility: Results were highly task-dependent:
- Positive: Functional classification improved by +32.5 pp F1-score.
- Negative: Security classification dropped by -4.5 pp, and Defects dropped by -7.8 pp.
Conclusion: Optimizing prompts only for diversity (via cosine distance) does not guarantee better classification utility and can be detrimental depending on the task.

RQ3: Synthetic vs. Human Data

Synthetic data surpassed human-authored data in Security (+7.8 pp F1) and Defects (+15.4 pp F1) tasks.
Synthetic data matched human data in Quality tasks (though with lower F1, it had higher precision).
Human data remained superior only in Functional classification (+9.5 pp), likely because the real dataset was larger and more balanced to begin with.

5. Significance and Implications

Practical Efficiency: Multi-sample prompting offers a "free" boost in diversity and performance without complex infrastructure, while also reducing API costs.
Caution on Automation: The study warns against blind application of automated prompt optimization (like PACE) that targets diversity in isolation. Scoring functions must be utility-aware (balancing diversity with task-specific signals) to avoid degrading model performance.
Scarcity Solution: In domains with scarce, imbalanced, or privacy-restricted data, synthetic generation can effectively replace or augment human data, potentially outperforming it by generating balanced, large-scale datasets on demand.
Generalizability: While tested on RE, the findings regarding the tension between diversity and utility apply broadly to any domain using LLMs for training data generation (e.g., healthcare, legal).

6. Limitations

Model Specificity: Results are tied to a specific LLM (gpt-4.1-nano); performance may vary with other models.
Scoring Function: PACE used a purely diversity-based scoring function (cosine distance); a utility-aware function might yield better results.
Domain Scope: Evaluation was limited to English RE tasks; validation in other domains/languages is needed.
Sample Size: The number of experimental runs (5) and real dataset sizes were relatively small.