PRISM: Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis

The Art of the Fine Print

Imagine you are teaching a robot to understand the world. For a long time, we've taught these robots using a simple game of "Question and Answer." You show them a picture of a cat and ask, "Is this a cat?" If they say "Yes," they get a gold star. This is how most AI models have learned to see: they are great at spotting things and answering direct questions about them.

But real life isn't just a series of simple questions. It's a messy bundle of instructions with hidden rules. Imagine you tell a human, "Write a slogan for this poster." You might also have a secret list of requirements: "It must mention the brand," "It cannot lie about the product," and "It should sound exciting." If the slogan is exciting but lies about the product, a human knows it's a failure. But for an AI, that "bundle" of rules is often too confusing. They tend to focus on the main request and ignore the fine print. This paper tackles that specific problem: teaching AI models to not just answer a question, but to act like a strict editor who checks every single rule before giving a final verdict.

The Paper: PRISM

The researchers behind this study, working with models from ByteDance and Peking University, noticed that current AI models are like students who only study for the final exam but ignore the homework instructions. They are great at answering "What is this?" but terrible at following complex, multi-step instructions where some rules matter more than others. To fix this, they created a new training method called PRISM (Priority-aware Rubric Internalization via Structured Multimodal Data Synthesis).

Think of PRISM as a "Rule-Making Factory." Instead of just showing the AI a picture and asking a question, the researchers built a system that generates a structured "rubric"—a checklist of rules—for every image. They didn't just ask the AI to guess the rules; they taught it to act as an executor. The model has to look at an image, read a list of rules (like "Check if the shoes are red" or "Ensure the text is polite"), and then verify each one individually before giving a final "Pass" or "Fail" judgment.

The team used a clever four-step process to create this training data. First, they invented a "persona" for the AI, like a "Brand Poster Reviewer" or a "Math Test Checker." Then, they guided the AI to generate a list of rules based on that persona. They filtered out the bad rules (the ones that were confusing or contradictory) and kept only the high-quality ones. Finally, they taught the model how to write a report that checks every rule one by one and then combines those checks into a final decision.

The results suggest that this method works incredibly well. When they trained a model called Qwen3-VL-4B with just 10,000 of these synthesized examples, its ability to follow these complex rules jumped from a lowly 9.5% to a much stronger 30.1%. This is a big deal because, even before this training, the strongest AI models in the world were struggling to get past that 30% mark. The paper suggests that the problem wasn't that the AI wasn't "smart" enough; it was that it hadn't been taught the specific skill of checking a list of prioritized rules.

Why It Matters

The most interesting finding is that this training didn't make the AI "dumber" at other tasks. Usually, when you teach a model a new, specific trick, it might forget how to do its old tricks. But here, the model got better at following rules without losing its general ability to understand images and text.

The authors also tested whether they could just "guide" the model into doing this during the test phase by giving it a few examples (like a reference sheet). They found that while giving examples helped a little, it wasn't nearly as good as actually training the model to internalize the process. It's the difference between giving a student a reference sheet during a test versus teaching them how to study so they actually understand the material.

In short, the paper suggests that to make AI truly helpful in the real world—where instructions are rarely simple—we need to stop treating them like question-answering machines and start training them like careful editors who respect the fine print.