When Generic Prompt Improvements Hurt: Evaluation-Driven Iteration for LLM Applications
This technical report introduces the Minimum Viable Evaluation Suite (MVES) framework and demonstrates through local ablation studies that generic prompt improvements can degrade specific LLM application performance, thereby advocating for evaluation-driven iteration to mitigate regression risks before deployment.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Idea: Why "Good" Prompts Can Sometimes Be Bad
Imagine you are training a very smart, but slightly unpredictable, robot assistant. You want it to do three specific jobs:
- The Accountant: Extract specific numbers from messy invoices and put them into a neat spreadsheet.
- The Librarian: Answer questions using only the books on a specific shelf, citing exactly which page the info came from.
- The Receptionist: Follow strict formatting rules (like "only say yes or no") and know when to say "I don't know."
The paper asks a simple question: If we give the robot a generic "be more helpful and polite" instruction, will it get better at all three jobs?
The surprising answer from the research is no. In fact, making the robot "nicer" or giving it generic advice often breaks its ability to do the specific jobs.
The Core Problem: The "One-Size-Fits-All" Trap
In traditional software (like a calculator), if you type 2 + 2, you always get 4. You can test it easily.
But Large Language Models (LLMs) are more like improvisational actors. If you ask them the same question twice, they might give slightly different answers. They are sensitive to how you phrase things.
The author argues that developers often make a mistake: they think adding a "generic improvement" to the robot's instructions (like "Be concise and accurate") is a free upgrade. They assume it helps everything.
The Analogy:
Imagine you are coaching a basketball team.
- The Accountant is the Center who needs to stay in the paint and grab rebounds.
- The Librarian is the Point Guard who needs to pass the ball exactly to the open player.
- The Receptionist is the Referee who needs to blow the whistle strictly.
If you tell the whole team, "Just be more helpful and energetic!" (a generic prompt improvement):
- The Center might start running up the court to help, leaving the basket undefended (breaking the strict format).
- The Point Guard might try to dribble too much and forget to pass (ignoring the source material).
- The Referee might get too friendly and forget to call fouls.
The paper shows that generic advice often helps one player while hurting the others.
The Experiment: Testing the "Generic" Myth
The author set up a small, controlled experiment (like a science fair project) to prove this.
- The Setup: They used two different robot models (Llama 3 and Qwen 2.5) and tested them on 30 specific scenarios for each of the three jobs (Accountant, Librarian, Receptionist).
- The Test: They tried five different versions of instructions:
- Version A (Baseline): Just the basic job description.
- Version B: Added a short "be helpful" wrapper.
- Version C: Added generic rules to the user's prompt (e.g., "Always be polite").
- Version D: A full "improved" prompt.
- Version E: A version that tried to be helpful without conflicting with the rules.
- The Results:
- For the Accountant (Extraction): The "improved" prompts worked great! The robot finally stopped chatting and just gave the numbers.
- For the Librarian (RAG): The "improved" prompts were a disaster. When the robot was told to be "helpful" or follow "generic rules," it started making things up or forgetting to cite its sources.
- Specifically: One robot (Qwen 2.5) went from getting 26 out of 30 answers right to only 9 out of 30 right just because a generic rule was added.
- For the Receptionist (Instructions): The results were mixed; some rules helped, others confused the robot.
The Solution: The "Minimum Viable Evaluation Suite" (MVES)
Because you can't guess what a prompt change will do, the author proposes a new way of working called MVES.
Think of MVES as a safety checklist before you launch a new feature. Instead of guessing, you must:
- Define the Failure: What exactly could go wrong? (e.g., "The robot forgets to cite sources.")
- Create a Test Set: A small, curated list of "Golden Cases" (like 30 specific questions) that you know the robot should answer perfectly.
- Run the Test: Every time you change the robot's instructions, you run these 30 cases.
- Check the Score: If the score drops on any job, you don't launch the change, even if it looks good on paper.
The Metaphor:
Imagine you are a chef. You want to add a new spice to your soup.
- Old Way: You taste the soup, think "It smells good," and serve it to 1,000 people.
- MVES Way: You have a "Taste Test Panel" of 30 specific dishes. You add the spice, taste the 30 dishes, and check: Did the spice ruin the dessert? Did it make the salad too salty? If the dessert is ruined, you don't serve the soup, even if the main course tastes amazing.
Key Takeaways for Everyone
- Don't Assume "More is Better": Adding generic instructions to an AI doesn't automatically make it smarter. It often breaks specific, strict tasks.
- Regression Testing is Essential: Just like you test a car's brakes after changing the engine, you must test an AI's specific tasks after changing its instructions.
- One Job, One Rule: A prompt that makes an AI great at writing creative stories might make it terrible at following strict data rules. You need to test them separately.
- The "Local" Lesson: The author admits this was a small test on a local computer. It's not a rule for every AI in the world, but it proves that prompt changes are risky experiments, not magic fixes.
In a Nutshell
This paper is a warning label for AI developers. It says: "Stop assuming that generic 'helpfulness' instructions will fix everything. They might fix one problem while breaking three others. Always run your specific test cases before you hit 'Deploy'."
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.