Instruction Stacking Collapse: A Benchmark and the Capability-Dependent Value of Prompt Compilation
This paper introduces a benchmark demonstrating that instruction-following performance in large language models degrades non-linearly as constraints accumulate due to pairwise conflicts, and shows that a training-free prompt compiler effectively mitigates this issue for weaker models while leaving stronger models largely unaffected.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to give a set of instructions to a very smart, but slightly literal-minded robot assistant. You know that if you ask it to do one thing, like "write a story," it will probably do a great job. But what happens if you pile on twenty different rules at once? "Use exactly 50 words," "format it as a JSON file," "include three citations," "don't use the letter 'e'," "start with a summary," and "make it sound like a pirate"? This is the world of Large Language Models (LLMs). These are the AI brains behind many chatbots and tools we use today. They are trained to follow human commands, but they aren't perfect. Scientists have long known that when you give them too many rules, they start to mess up. But nobody really knew how badly they would fail, which rules fight against each other, or if there was a way to fix it without retraining the robot's brain from scratch. This paper dives into that messy pile of rules to see exactly where the robot gets confused and how we might help it sort things out.
The Great Instruction Stack Collapse
Think of giving an AI a prompt like handing a chef a recipe card. If the card says "Make a sandwich," the chef is happy. But imagine if the card suddenly had a sticky note on top saying "Use only red ingredients," another saying "No bread allowed," a third saying "It must be served in a blue bowl," and a fourth screaming "Do not use your hands!" If you stack twenty of these conflicting notes on top of the recipe, the chef doesn't just get confused; they often give up on half the rules entirely. They might make a sandwich, but they forget the red ingredients. They might use the blue bowl, but they ignore the "no bread" rule. And the scary part? The chef doesn't tell you they failed. They just serve you a weird sandwich and say, "Here you go."
This paper, titled "Instruction Stacking Collapse," is a deep dive into exactly how this happens. The researchers built a giant playground where they tested how well three different top-tier AI models (Claude Sonnet 4.6, GPT-5-mini, and Gemini 2.5 Flash) could handle a growing pile of instructions. They started with just one rule and slowly added more, up to twenty rules at once.
The Crash: When Rules Fight Each Other
The results were dramatic. When the AI had just one instruction to follow, it was a superstar, getting it right about 96% of the time. But as the pile of rules grew to twenty, the performance crashed.
- The strongest model (Claude Sonnet) dropped to about 60% success.
- The middle model (Gemini) fell to 43%.
- The weakest model (GPT-5-mini) hit a wall, managing only 20% of the instructions correctly.
It wasn't a slow, boring decline. It was a "collapse." The researchers found that the rules weren't just getting lost; they were actively fighting each other. They discovered a "conflict map." For example, if you tell the AI to "output valid JSON" (a specific computer code format) and also tell it to "use markdown headers" (like ##Title), the AI gets stuck. It can't do both. If it chooses JSON, it silently breaks the markdown rule. If it chooses markdown, it breaks the JSON rule. The paper found that the instruction "output JSON" was the biggest troublemaker, causing conflicts with nine other common rules.
The Magic Fix: The Instruction Compiler
So, is the solution to just give the AI fewer rules? Not necessarily. The researchers asked: "Can we fix the prompt before the AI sees it, without teaching the AI anything new?"
They invented a tool called an Instruction Compiler. Think of this compiler as a super-organized assistant who looks at your messy pile of twenty sticky notes before you hand them to the chef. The compiler does three things:
- Groups them: It puts all the "format" rules together and all the "length" rules together.
- Merges them: If two rules say the same thing, it combines them into one clear sentence.
- Prioritizes them: If two rules fight (like "use JSON" vs. "use markdown"), the compiler adds a note saying, "Hey, if these two clash, do the JSON one first."
This compiler runs once, creates a clean, organized checklist, and then that same checklist is used for every single question the AI answers. It's like giving the chef a single, perfectly written recipe card instead of a messy stack of notes.
The Surprise: Who Gets Helped?
Here is the most interesting part of the story. The researchers expected this "clean recipe card" to help everyone equally. But it didn't. The benefit depended entirely on how "smart" the chef (the AI model) was.
- For the weaker models (like GPT-5-mini): The compiler was a lifesaver. It boosted their success rate by +11 percentage points. It turned a model that was failing 80% of the time into one that was failing only 69% of the time. This is huge because these weaker models are the ones used most often in the real world because they are faster and cheaper.
- For the strongest models (like Claude Sonnet): The compiler did almost nothing. In fact, it slightly hurt their performance (by about -1.2 points).
Why? The paper suggests that the strongest models are so smart that they can already look at a messy pile of twenty rules and figure out the structure, the conflicts, and the priorities all by themselves. They don't need the compiler to organize the notes for them; in fact, the extra notes just clutter their thinking. The weaker models, however, get overwhelmed by the mess. They need the compiler to act as a translator, turning the chaos into a clear, step-by-step plan.
What This Means for You
This research tells us two big things. First, when we ask AI to do complex things with many rules, we can't just throw a wall of text at it and expect it to work. The rules will collapse, and the AI will silently fail. Second, we don't need to build bigger, smarter robots to fix this. Sometimes, the solution is just better organization.
The paper proves that a simple, free tool that reorganizes instructions can rescue weaker AI models, making them much more reliable for the tasks we use them for every day. It's a reminder that in the world of AI, sometimes the smartest thing you can do isn't to make the robot smarter, but to make the instructions clearer. And for the super-smart robots? They might just prefer to figure it out on their own.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.