Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction
This paper introduces the Constraint Saturation Evaluation (CSE) benchmark to demonstrate that while large language models can follow individual constraints proficiently, their ability to satisfy multiple simultaneous constraints collapses multiplicatively beyond 5–6 constraints due to the independent accumulation of failures, particularly affecting structural reasoning over lexical details.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to bake a cake, but instead of just following one recipe, you are given a list of rules. "Use exactly 200 grams of flour," "Don't use any sugar," "The cake must be blue," and "It must be shaped like a star." If you only have one rule, it's easy. If you have two or three, you can probably manage. But what happens if someone hands you a list of fifty rules all at once? This is the world of Large Language Models (LLMs), the super-smart computer programs that write stories, answer questions, and solve problems. These models are like digital chefs who have read almost every book in the library. Scientists have known for a while that these chefs are great at following a single instruction. But nobody was sure what happened when you asked them to juggle a dozen instructions at the same time. Does their performance drop a little bit, like a tired runner slowing down? Or does it suddenly crash, like a house of cards collapsing when you blow on it? This question matters because as we start using these AI chefs to build real-world tools—like safety systems, legal documents, or complex code—we need to know exactly how many rules they can handle before they start making mistakes.
Enter a new study that treats these AI models like a science experiment in a giant kitchen. The researchers, led by Mariya Vasileva, built a special testing ground called Constraint Saturation Evaluation (CSE). Think of it as a "stress test" for the AI's brain. They didn't just ask the models to write a story; they gave them a specific number of rules to follow simultaneously, ranging from just one rule up to twelve. These rules were strict and unchangeable, like "Your response must have exactly 3 paragraphs," "No letter 'e' allowed," or "Every sentence must start with the letter 'A'." The researchers then watched to see how many times the AI could get every single rule right at the same time.
The results were surprising and a bit dramatic. The study found that AI models don't just get a little tired as you add more rules; they hit a tipping point. Imagine you are stacking blocks. For the first few blocks, the tower is stable. But once you reach a certain height—around 5 or 6 rules for even the smartest models—the tower doesn't just wobble; it suddenly collapses. The researchers discovered that while an AI might still get 40% of individual rules right when there are eight of them, the chance of it getting all eight right at once drops to less than 6%. It's like a musician who can play a single note perfectly, or even two, but as soon as you ask them to play a complex symphony with twelve different instruments at once, they forget the melody entirely.
The study also figured out why this happens. It turns out the rules don't fight each other like enemies in a game. Instead, they act like a chain reaction. If the AI messes up one small part of the answer—say, it forgets to count the sentences correctly—then every rule that depends on sentences fails at the same time. It's not that the rules are confusing each other; it's that the AI's "working memory" gets overloaded. The researchers found that rules requiring the AI to keep track of things over time (like counting words or maintaining a pattern) break down twice as fast as simple rules (like "don't use a specific word").
Perhaps the most interesting part is what the researchers tried to fix. They asked: "Can we help the AI by making it plan ahead? Or by letting it try again?" They tried giving the AI a "scratchpad" to plan its answer, or letting it correct its own mistakes. The result? It helped a tiny bit, maybe allowing the AI to handle one or two extra rules, but it didn't stop the crash. The only real solution, the paper suggests, is to make the AI better at following each individual rule in the first place. No amount of planning or retrying can fix the fact that the math of juggling too many things at once eventually leads to a drop in performance.
In the end, this paper draws a clear line in the sand. For the smartest AI models tested, the limit for reliable, multi-rule instruction following is roughly 5 to 6 constraints. Beyond that, the system becomes unreliable, not because the AI is "stupid," but because the sheer number of things it has to hold in its head at once creates a mathematical collapse. It's a reminder that even the most advanced digital chefs have a limit to how many recipes they can follow at once, and if we want them to do more, we need to build them stronger, not just give them more instructions.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.