Your Prompt Is Not the Only Prompt: How Much Do LLMs Weight Structured-Output Schema Descriptions?
This paper demonstrates that while system prompts generally remain the safest default for instruction placement, schema descriptions are a potent and model-dependent instruction channel that can override prompts, making unified schema design and empirical validation critical for optimizing structured-output performance.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a very smart, but slightly literal, robot how to sort a pile of mixed-up toys. You have two ways to give it instructions: you can talk to it directly (like a teacher giving a lecture), or you can hand it a special form with boxes to fill out, where the instructions are written right next to the boxes. This is the world of "Large Language Models" (LLMs), which are the brains behind many modern AI tools. Usually, people think the "talk" (the prompt) is where the real rules live, and the "form" (the structured output schema) is just a boring template for the robot to follow. But lately, developers have been wondering: does the robot actually read the notes written on the form, or does it just ignore them and listen only to the teacher? This question matters because if we put our most important rules in the wrong place, the robot might sort the toys completely wrong, breaking the apps and tools we rely on every day.
A researcher named Sin-Ying (Alina) Lin decided to settle this debate with a clever experiment. Instead of arguing about it, she set up a game where a robot had to sort short messages into four secret categories, using made-up names like "tomil" and "varek" so the robot couldn't guess the answers based on real-world knowledge. She then played a game of "musical chairs" with the instructions, moving the exact same definitions from the teacher's lecture (the system prompt) to the notes on the form (the schema description), and sometimes even putting them in the user's chat box.
What she found was a bit of a surprise: the robot's ears are not all the same size. For some models, like GPT-4.1 and GPT-5.4 (without special thinking modes), the teacher's voice was king. When the definitions were moved to the form, these robots got confused and their accuracy dropped by about 11 to 13 percentage points. They seemed to think, "The form is just for filling out; the real rules are in the speech!" However, for other models, like Claude Haiku 4.5, the form was actually the boss. When the instructions on the form were changed to be wrong, this robot's performance crashed from 52.5% accuracy down to a dismal 7%. It turned out that for some robots, the notes on the form are so loud that they drown out the teacher's correct instructions.
The paper also discovered a "magic trick" to fix the robots that ignored the form. By adding a mandatory step where the robot had to write down its reasoning in a box before it could pick the final answer, the performance jumped up by 15 to 24 percentage points. It was as if forcing the robot to "show its work" made it pay attention to the rules written on the form.
So, the big takeaway isn't that one place is always better than the other. Instead, the paper suggests that every robot model has its own personality and listens to different channels with different volumes. The safest bet for now is to keep your most important rules in the system prompt (the teacher's voice), but you must be careful not to write conflicting rules on the form, or some robots will follow the wrong ones. The real lesson is that you can't just guess; you have to test each robot to see where it listens best, and sometimes, simply changing the shape of the form to include a "thinking step" is a more powerful fix than rewriting the instructions entirely.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.