💬 NLP

\textsc{DiARC}: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models

This paper proposes \textsc{DiARC}, a method that enhances the ARC-like reasoning abilities of large language models by constructing preference pairs with informative negative samples through visual transformations, rule inversion, and task-specific editing, thereby enabling models to distinguish between correct and incorrect solutions more effectively.

Original authors: Yuxuan Yang, Feiyang Li, Yile Wang

Published 2026-06-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yuxuan Yang, Feiyang Li, Yile Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Teaching AI to Solve Puzzles

Imagine you are trying to teach a student how to solve a visual puzzle. You show them a few examples: "Here is a red square turning into a blue circle. Here is a green triangle turning into a yellow star." Then, you give them a new shape and ask, "What does this turn into?"

This is the ARC (Abstraction and Reasoning Corpus) challenge. It's a test of pure logic and pattern recognition. While big AI models (LLMs) are great at writing poems or solving math problems, they struggle with these visual puzzles. They often guess randomly or get stuck because they haven't learned the deep rule behind the change.

The Old Way: Only Showing the Right Answer

Previously, researchers tried to fix this by showing the AI only the correct answers.

The Analogy: Imagine a teacher showing a student a math problem and only saying, "The answer is 5." The student memorizes that "5" is the answer to that specific problem, but they don't understand why it's 5. If the problem changes slightly, the student fails.
The Paper's Critique: The authors argue that just showing the "right" answer isn't enough. The AI needs to learn what not to do.

The New Idea: Learning from "Near-Miss" Mistakes

The authors propose a new method called DIARC. Their core idea is simple: To learn the right rule, you must also learn to spot the wrong rules.

The Analogy: Think of a master chef teaching an apprentice.
- Old Way: The chef says, "This soup tastes perfect. Remember this taste."
- DIARC Way: The chef says, "This soup tastes perfect. But look at these three other bowls: one has too much salt, one is missing the herbs, and one is burnt. They all look like soup, but they are wrong. Can you tell the difference?"

By showing the AI the "almost right" but actually wrong answers, the AI learns to distinguish the true pattern from the fake ones.

How They Make the "Wrong" Answers (The Three Tricks)

The paper describes three clever ways to create these "wrong" answers (negative samples) without changing the original puzzle:

The Visual Tweak (Output-Level):
- What they do: They take the correct answer and mess it up slightly. Maybe they shift the whole picture a tiny bit to the left, or they blur the edges.
- The Metaphor: It's like taking a perfect photo and slightly tilting the camera or adding a little static noise. The photo still looks like the scene, but it's not exactly right.
The Rule Flip (DSL-Level):
- What they do: They look at the "code" or logic the AI uses to solve the puzzle. If the rule is "Move everything up," they flip it to "Move everything down."
- The Metaphor: If the rule for a game is "Jump when you see a red light," they create a fake rule: "Jump when you see a green light." It sounds logical, but it's the opposite of the truth.
The Smart Edit (Task-Specific):
- What they do: They use a super-smart AI (like GPT-5.4) to rewrite the rule for a specific puzzle to be the "semantic opposite."
- The Metaphor: If the puzzle is about "filling a bucket," the AI edits the rule to be about "emptying the bucket." It's a very specific, tricky mistake that looks like it could be right but isn't.

The Result: A Smarter Student

The researchers tested this method on six different puzzle benchmarks. They used three different open-source AI models and compared them to models that only learned from correct answers.

The Outcome: The models trained with DIARC (the "mistake-spotting" method) got significantly better scores.
The Highlight: Using a model called Qwen3, they achieved over 96% accuracy on some of the hardest puzzles. This beat many expensive, closed-source models and even specialized AI designed just for these tasks.

Summary

The paper argues that to make AI better at abstract reasoning, we shouldn't just feed it the right answers. We need to teach it to reject the wrong ones. By creating "near-miss" mistakes and training the AI to choose the right path over the tempting wrong paths, the AI learns the true logic behind the puzzles much faster and more effectively.

Technical Summary of DIARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models

Problem Statement

The Abstraction and Reasoning Corpus (ARC) requires models to infer latent transformation rules from a small number of input-output grid examples and apply them to new test inputs. While Large Language Models (LLMs) have shown progress in natural language and mathematical reasoning, they struggle with ARC-like abstract symbolic tasks. Current approaches to improve LLM performance on ARC primarily rely on Supervised Fine-Tuning (SFT) using data augmentation to create more training examples. However, these methods generally yield unsatisfactory results with open-source models and are costly with closed-source ones.

The authors identify three key limitations in current SFT-based approaches:

Generative Difficulty: ARC output sequences are often long, making step-by-step token prediction difficult.
Local vs. Global: SFT focuses on local token information, whereas ARC tasks require understanding global grid characteristics.
Lack of Negative Guidance: Existing methods rely solely on positive samples (correct input-output pairs) and lack guidance from incorrect outputs. The authors argue that solving ARC requires not just learning correct patterns but also the ability to distinguish correct rules from plausible but incorrect "near-miss" alternatives.

Methodology: DIARC

To address these limitations, the authors propose DIARC, a preference alignment framework that optimizes LLMs to distinguish between correct outputs and rejected negative samples. The method consists of three main components:

1. Negative Sample Construction
Instead of relying solely on correct targets, DIARC constructs preference pairs $(x, y^+, y^-)$ where $y^+$ is the correct output and $y^-$ is a negative sample derived from the same task. The negative samples are constructed at three distinct levels to ensure they are informative "near-misses":

Output-Level Visual Transformations ( $NG_{Vis}$ ): Directly modifies the output grid using 28 atomic operations across four families:
- Grid-block operators: (e.g., pooling, mode-centering, removing small components) that preserve coarse layout but damage fine-grained structure.
- Rigid shifts: Shifting grid content in eight directions with 10% or 30% magnitude.
- Morphological/Geometry operators: (e.g., erosion, dilation, skeletonization) that alter object shapes while keeping spatial layout.
- Local perturbations: Injecting noise or swapping boundary pixels.
DSL-Level Rule Inversion ( $NG_{DSL}$ ): Operates at the program level. The authors analyze task-specific transformation programs written in a Domain-Specific Language (DSL). They identify recurring local rule patterns and apply a semantic reversal (inversion) to these patterns, generating a program that produces a rule-incorrect output while remaining close to the original logic.
Task-Specific Rule Editing ( $NG_{Edit}$ ): Uses an LLM (GPT-5.4) to assist in editing the transformation rule for a specific task. The goal is to generate an alternative rule that is the semantic opposite of the original or a close variant that leads to a different transformation behavior, ensuring the negative is structurally similar but logically distinct.

2. Preference Optimization Pipeline
Once preference pairs are constructed, the model is trained using Direct Preference Optimization (DPO). Unlike Reinforcement Learning from Human Feedback (RLHF), which requires an explicit reward model, DPO directly optimizes the policy model $\pi_\theta$ to assign higher likelihood to the chosen output ( $y^+$ ) than the rejected output ( $y^-$ ) relative to a reference model $\pi_{ref}$ . The loss function is:
$L_{DPO} = -E \left[ \log \sigma \left( \beta \log \frac{\pi_\theta(y^+|x)}{\pi_{ref}(y^+|x)} - \beta \log \frac{\pi_\theta(y^-|x)}{\pi_{ref}(y^-|x)} \right) \right]$
The policy model is initialized from an SFT model, and the reference model is a frozen copy of that SFT model.

Key Contributions

Shift in Supervision Paradigm: The paper argues that ARC reasoning improvement requires moving beyond pure positive sample supervision to include the discrimination of negative samples, drawing on cognitive science principles that learning involves identifying and correcting errors.
Three-Tier Negative Construction: The proposal of a systematic method to generate informative negative samples via visual transformations, DSL rule inversion, and task-specific rule editing.
Preference Alignment for ARC: The application of DPO to ARC tasks, demonstrating that optimizing for the distinction between correct and plausible incorrect outputs enhances reasoning capabilities.

Experimental Results

The authors evaluated DIARC on six benchmarks: ARC-AGI-1, ARC-AGI-2, MiniARC, ConceptARC, 1D-ARC, and ARCcommunity, using three open-source LLM backbones (Llama-3.2-3B, Mistral-NeMo-Minitron-8B, and Qwen3-4B).

Consistent Improvements: DIARC consistently outperformed baseline SFT-only models across all datasets and model sizes.
State-of-the-Art Performance: Using the Qwen3 model, DIARC achieved 97.00% accuracy on ARC-AGI-1 and 33.06% on ARC-AGI-2. On MiniARC and ConceptARC, it achieved 97.32% and 97.29% respectively.
Comparison: These results outperform various ARC-specialized models (e.g., HRM, GRAM, VARC) and strong closed-source LLMs (e.g., GPT-4, o1-mini, Kimi K2.5).
Ablation: Experiments showed that all three negative construction strategies contributed to performance gains, with "Random Perturb" and "Rigid Shift" often providing significant improvements on specific benchmarks.

Significance and Claims

The paper claims that the ability to distinguish between correct rules and plausible near-miss alternatives is a crucial skill for ARC reasoning that is often overlooked in standard SFT. By leveraging preference alignment, DIARC enables models to learn not just what the answer is, but why certain plausible alternatives are incorrect. The authors state that this approach provides a cost-effective way to improve open-source models to levels competitive with or exceeding expensive closed-source systems on abstract reasoning tasks, without requiring the massive inference-time costs associated with frontier models. The work suggests that future improvements in ARC reasoning may depend more on the quality of negative sample construction and preference optimization than on simply scaling up data or model size.