🤖 machine learning

Where Not to Learn: Prior-Aligned Training with Subset-based Attribution Constraints for Reliable Decision-Making

This paper proposes a training method that aligns models with human priors by penalizing reliance on off-prior evidence through subset-based attribution constraints, thereby improving both task accuracy and decision reasonability in image classification and GUI agent tasks.

Original authors: Ruoyu Chen, Shangquan Sun, Xiaoqing Guo, Sanyi Zhang, Kangwei Liu, Shiming Liu, Zhangcheng Wang, Qunli Zhang, Wei Wang, Hua Zhang, Xiaochun Cao

Published 2026-07-15

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ruoyu Chen, Shangquan Sun, Xiaoqing Guo, Sanyi Zhang, Kangwei Liu, Shiming Liu, Zhangcheng Wang, Qunli Zhang, Wei Wang, Hua Zhang, Xiaochun Cao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Detective's Dilemma: Why Being Right Isn't Enough

Imagine you are teaching a robot to recognize a cat. You show it a million pictures of cats, and eventually, the robot gets really good at saying "Meow!" whenever it sees a fluffy shape. But here's the catch: the robot isn't actually looking at the cat's whiskers or ears. Instead, it has learned a sneaky shortcut. It noticed that in all your training photos, the cats were sitting on a specific brand of red rug. So, the robot ignores the cat entirely and just looks for the red rug. If you show it a cat on a blue sofa, it gets confused. If you show it a red rug with no cat, it might still scream "Meow!"

This is a common problem in the world of Artificial Intelligence, specifically in a field called machine learning. Scientists build these "models" to make decisions, but often, they learn to cheat by finding easy patterns (like the red rug) rather than understanding the real reason for a decision (the cat). This is dangerous because if the robot is driving a car or clicking buttons on a screen, it needs to know why it's doing something, not just guess based on a lucky coincidence. To fix this, researchers use "human priors"—which is just a fancy way of saying "what a human thinks is important." If a human points to a cat and says, "Look here," the robot should learn to look there too. But getting the robot to agree with the human is tricky, because the robot's brain works very differently from ours.

The Paper's Big Idea: "Don't Learn Where You're Not Supposed To"

This paper, titled "Where Not to Learn," tackles the problem of robots cheating by using a clever, selective training method. The authors suggest that instead of forcing the robot to look only at the right place (which can be too strict and confuse the robot), we should simply punish it when it looks at the wrong place.

Think of it like a teacher grading a student's essay. If the student writes a brilliant paragraph about the wrong topic, the teacher doesn't erase the whole essay. Instead, the teacher puts a big red circle around the wrong part and says, "This doesn't belong here." The student then learns to focus their energy on the right topic. The authors call this "Prior-Aligned Training." They use a special tool called "subset-selection attribution" to act as the teacher's magnifying glass. This tool helps the robot figure out exactly which tiny pieces of an image or screen are making it make a decision.

Here is how the magic happens:

The Magnifying Glass: The tool looks at the robot's decision and highlights the top few "clues" it used.
The Check: The system compares these clues to the "human prior" (the area a human said was important, like a bounding box around a button).
The Gentle Nudge: If the robot's top clue is inside the human's box, the system says, "Good job!" and moves on. But if the robot's top clue is outside the box (like looking at the red rug instead of the cat), the system applies a penalty. It tells the robot, "Stop relying on that clue."

The paper explicitly argues against older methods that tried to "suppress" or silence all non-human areas at once. The authors found that this "hard suppression" is too blunt; it's like telling a student to stop thinking about anything except the answer key, which often makes them forget how to solve the problem entirely. Instead, their method is "selective." It only intervenes when the robot's most important evidence is wrong. If the robot is already looking at the right spot, the system leaves it alone, letting other parts of the image contribute freely.

What They Found: Smarter Robots, Better Decisions

The researchers tested this idea in two very different worlds:

Image Classification: Teaching computers to identify objects in photos (like boats or animals).
GUI Agents: Teaching AI to click buttons on a computer screen (like a virtual assistant setting an alarm).

In both cases, the results were promising. When they used their "selective penalty" method, the models didn't just get better at following the rules; they actually got better at the main task too.

On the ImageNet-S dataset (a collection of images with specific object masks), their method improved the accuracy of a model called ViT by 4.95 points and a model called ResNet-101 by 1.74 points.
More importantly, the "reasonableness" of the decisions improved. They measured this using a "Point Game," which checks if the AI is looking at the right object. On the Saliency-Bench dataset, their method boosted the Point Game score for the ViT model from 0.4363 to 0.5463.

The paper suggests that by forcing the AI to rely on the "right" evidence (the human-prior regions), it becomes more robust. When they tested the models with noisy, messy images (adding random static to the pictures), the models trained with this method held up better than the standard ones. This implies that by stopping the AI from learning "cheat codes" (like the red rug), it learns to rely on stable, meaningful features that work even when things get messy.

The authors also ran "ablation studies," which are like taking the engine apart to see which part does what. They found that the "Deviation Loss" (the penalty for looking at the wrong place) was the heavy lifter, driving most of the improvements in both accuracy and reasonableness. The "Redundancy Loss" (which stops the AI from over-relying on multiple wrong clues) helped a little bit more with the "reasonableness" score but didn't change the accuracy as much.

In short, the paper shows that you don't need to micromanage an AI's every thought. You just need to catch it when it's looking in the wrong direction and gently steer it back. This approach makes AI not only more honest about why it makes a decision but also more accurate and reliable in the real world. The authors conclude that this method works across different types of AI, from simple image classifiers to complex "agents" that can click buttons and follow instructions, suggesting a path toward AI that is both smart and sensible.

Technical Summary: "Where Not to Learn: Prior-Aligned Training with Subset-based Attribution Constraints for Reliable Decision-Making"

Problem Statement

Reliable machine learning models must not only predict correctly but also base their decisions on acceptable, task-relevant evidence. However, conventional supervised learning relies solely on class-level labels, allowing models to exploit "shortcut correlations" (e.g., background textures or spurious features) rather than intended causal evidence. While human priors (such as bounding boxes, target interface elements, or sparse clicks) can theoretically constrain this behavior, aligning model decision evidence with these priors remains challenging. Existing methods often fail because:

Faithfulness Gap: They rely on post-hoc attribution methods (e.g., Grad-CAM, LIME) that may not accurately reflect the true decision evidence.
Over-regularization: They enforce hard, uniform suppression of all non-prior regions, ignoring that different regions contribute unequally to the decision and potentially stifling valid learning signals.

Methodology

The authors propose a selective prior-constrained training framework that uses subset-selection-based attribution as a training-time signal to guide model behavior.

1. Attribution Mechanism:
Instead of gradient-based or attention-based explanations, the method employs subset-selection-based attribution (specifically LIMA for image classification and EAGLE for MLLM-based GUI agents). This approach identifies compact, decision-supporting subsets of input regions by iteratively selecting regions that maximize a set function $F(\cdot)$ , ranking them by influence. This method is chosen for its higher faithfulness in identifying the true evidence driving predictions.

2. Selective Alignment Principle:
The core innovation is an asymmetric alignment strategy. The model is not forced to align with human priors at all times. Instead, constraints are applied only when the model's most salient attributed evidence deviates from the human prior region.

If the top-attributed evidence overlaps with the human prior, no penalty is applied, allowing other regions to contribute freely.
If the top-attributed evidence lies outside the prior, the model is penalized for relying on "off-prior" cues.

3. Loss Functions:
The training objective combines standard task loss ( $L_{task}$ ) with two specific alignment losses applied intermittently (only when the model predicts correctly and at specific training intervals):

Deviation Loss ( $L_{deviation}$ ): Penalizes the utility score of the top-ranked attribution region ( $v_{\pi1}$ ) if its spatial overlap with the human prior $H$ falls below a threshold $\tau$ . This discourages reliance on the most influential off-prior evidence.
Redundancy Loss ( $L_{redundancy}$ ): Regulates higher-order attribution regions (ranks $r \geq 2$ ). If these regions are off-prior, their marginal gains ( $\Delta_{i,r}$ ) are suppressed via ReLU. This prevents the accumulation of spurious evidence from multiple unintended regions.

4. Training Procedure:
The algorithm alternates between standard task supervision and evidence-level alignment. Alignment losses are computed only for correctly predicted samples at regular intervals (e.g., every 10 steps for classification, every 5 for GUI agents) to reduce computational overhead and prevent over-constraining the model during early learning phases.

Key Contributions

Faithful Attribution for Training: The paper demonstrates that subset-selection-based attribution (LIMA/EAGLE) provides a more reliable training signal for human-prior alignment than conventional gradient- or activation-based explanations.
Selective Prior-Constrained Objective: A novel loss formulation that intervenes only when top-attributed evidence deviates from human priors, avoiding the pitfalls of hard, uniform suppression of non-prior regions.
Cross-Task Validation: The framework is validated across two distinct settings:
- Discriminative Classification: Image classification on ImageNet-S and Saliency-Bench.
- Autoregressive Decision-Making: Click decisions in Multimodal Large Language Model (MLLM)-based GUI agents.

Experimental Results

The method was evaluated against baselines including RRR (gradient suppression), XIL (feature-level Grad-CAM suppression), and MEGL (Grad-CAM alignment).

Image Classification:
- Accuracy: The method consistently improved Top-1 accuracy across backbones (CLIP, ViT, ResNet). For example, on Saliency-Bench with ViT, accuracy rose from 0.5150 (Fine-tuning) to 0.5694.
- Decision Rationality: Measured by the Point Game (evaluating attention to target objects), the method showed significant gains. On Saliency-Bench with ViT, the Point Game score improved from 0.4363 to 0.5463.
- Conditional Reliability: When conditioning on successful Point Game outcomes (i.e., when the model attends to the correct object), the method yielded substantial accuracy gains (e.g., CLIP on ImageNet-S improved from 0.7093 to 0.8377), indicating that correct evidence leads to more reliable predictions.
- Robustness: The method demonstrated superior robustness against Gaussian noise compared to standard fine-tuning.
GUI Agent Tasks:
- Applied to an MLLM-based agent (AgentCPM-GUI) for click decisions. The method improved the consistency between the model's "thinking" (reasoning traces) and the executed decision, enhancing decision rationality without sacrificing task performance.

Significance and Claims

The paper claims that constraining decision evidence using faithful attribution signals yields models that are not only more interpretable but also more robust and effective. By shifting the focus from "what" the model predicts to "where" it looks, the method addresses the root cause of shortcut learning. The authors emphasize that their approach does not propose a new paradigm of explanation alignment but rather instantiates a practical, selective alignment strategy that leverages high-fidelity attribution to improve causal reasonability and decision reliability across both discriminative and generative settings. The results suggest that models trained with these constraints are less likely to fail unpredictably in safety-critical or interactive environments.