💻 computer science

GIFT: Generalizing Intent for Flexible Test-Time Rewards

The paper presents GIFT, a framework that leverages language models to infer high-level human intent from demonstrations, enabling reward functions to generalize robustly to novel environments and objects without retraining by mapping test states to behaviorally equivalent training states based on intent rather than surface-level cues.

Original authors: Fin Amin, Nathaniel Dennler, Andreea Bobu

Published 2026-03-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fin Amin, Nathaniel Dennler, Andreea Bobu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to pack a bag for an art class. You show it a few times: "Put the paintbrush in the bag. Put the sketchbook in the bag."

The robot learns this. But here's the problem: if you later ask the robot to pack for a different art session, and you hand it a lump of molding clay, a standard robot might get confused.

Why? Because the robot didn't actually learn the concept of "art supplies." Instead, it learned a silly rule like: "If it looks like a paintbrush, put it in the bag." So, when it sees a dish scrubber (which looks a bit like a brush) or a toothbrush (which has the word "brush" in its name), it might try to pack those instead of the clay. It's following the surface details, not the real goal.

This paper introduces a new framework called GIFT (Generalizing Intent for Flexible Test-Time rewards) to fix this.

The Core Idea: The "Intent Detective"

Think of GIFT as a robot that doesn't just memorize pictures; it acts like a detective trying to figure out your motive.

The Old Way (Visual/Language Similarity):
- Visual: The robot sees a dish scrubber and a paintbrush. They look similar (both have bristles). The robot thinks, "They are the same!" and packs the scrubber.
- Language: The robot reads "toothbrush" and "paintbrush." They both end in "brush." The robot thinks, "They are the same!" and packs the toothbrush.
- Result: The robot fails because it's looking at the wrong clues.
The GIFT Way (Intent Similarity):
- GIFT uses a large language model (like a super-smart AI chatbot) to act as a translator.
- It looks at what you did (putting art stuff in a bag) and what you didn't do (leaving it on the table).
- It asks the AI: "What is the human's real goal here?"
- The AI answers: "The goal is 'Pack Art Supplies'."

How It Works: The "Shape-Shifting" Trick

Once GIFT knows the goal is "Pack Art Supplies," it performs a magic trick called Intent-Conditioned Alignment.

Imagine the robot is looking at a table full of new objects it has never seen before: a lump of clay, a screwdriver, and a paintbrush.

Without GIFT: The robot sees the clay and thinks, "I've never seen this. I don't know what to do."
With GIFT: The robot asks the AI, "Under the rule 'Pack Art Supplies,' what is this lump of clay equivalent to?"
- The AI says, "Ah, under that rule, the clay is the same as the paintbrush you saw in training."
- The AI also says, "The screwdriver is a distractor; it's not art."

GIFT then mentally swaps the clay for the paintbrush in its memory. It tells the robot: "Treat this clay exactly like you treated that paintbrush."

Now, the robot can use the exact same instructions it learned yesterday to handle a completely new object today, without needing to be retrained.

The Real-World Test

The researchers tested this on a real robot arm (a Franka Panda) and in a computer simulation.

The Setup: They taught the robot to pack "art supplies" or "valuable items" using a few examples.
The Test: They gave the robot new objects it had never seen (like molding clay or a specific iPhone model) and mixed in "trick" objects (like a dish scrubber or a paper ring).
The Result:
- The old methods (looking at pictures or words) got tricked by the dish scrubbers and paper rings.
- GIFT correctly identified that the clay was "art" and the iPhone was "valuable," ignoring the trick objects. It won the game almost every time.

The Big Takeaway

Most robots are like students who memorize the answers to a specific test. If you change the questions slightly, they fail.

GIFT teaches the robot to understand the subject matter. It's the difference between memorizing that "2 + 2 = 4" and understanding the concept of addition. Once you understand the concept, you can solve "2 + 2" even if the numbers are written in a different font, or if you're adding "apples" instead of "numbers."

By focusing on human intent (the "why") rather than surface features (the "what it looks like"), GIFT allows robots to be flexible, smart, and ready for the real world.

1. Problem Statement

Robots typically learn reward functions from user demonstrations (e.g., via Inverse Reinforcement Learning). However, these learned rewards often fail to generalize to new environments or objects encountered at test time.

The Core Issue: Learned rewards tend to latch onto spurious correlations in the training data (surface-level visual or linguistic features) rather than capturing the user's high-level intent.
Example: A robot trained to "pack a paintbrush" might learn to pack any brush-like object (visual similarity) or any object with the word "brush" in its name (linguistic similarity). It fails to recognize that "molding clay" is the correct object to pack if the intent is "pack art supplies," because clay does not look like a brush or share the word "brush."
Consequence: When distribution shifts occur (new objects, layouts, or contexts), the robot either ignores critical task aspects or fixates on irrelevant ones, leading to catastrophic failure. Existing methods relying on visual or semantic similarity baselines diverge from what humans actually care about regarding task success.

2. Methodology: The GIFT Framework

GIFT (Generalizing Intent for Flexible Test-Time rewards) is a framework designed to ground reward generalization in human intent rather than surface cues. It operates in two main stages using Large Language Models (LLMs) as reasoning engines, without requiring retraining of the reward function.

A. Inferring High-Level Intent

Input: The system takes pairs of human-preferred demonstrations ( $D_H$ ) and non-preferred reference trajectories ( $D_{\neg H}$ ) from the same scene.
Process: An LLM ( $J$ $J$ ) is prompted to contrast these behaviors to infer a natural language summary of the user's underlying intent ( $\hat{\omega}$ $\overset{ω}{^}$ ).
- Example: Instead of "avoid coffee on laptops," the LLM infers "avoid liquids near water-sensitive electronics."
Output: A high-level intent description that abstracts away specific object instances.

B. Intent-Conditioned State Similarity & Alignment

Once the intent $\hat{\omega}$ is inferred, GIFT uses it to define a new notion of state similarity.

Intent-Conditioned Kernel: Instead of using visual (e.g., DINO embeddings) or linguistic (e.g., BERT embeddings) similarity, GIFT uses an LLM to act as an implicit similarity function $K(s, s' | \hat{\omega})$ .
Alignment Operator ( $f_{\hat{\omega}}$ ): For a novel test state $s'$ $s^{'}$ , the system queries the LLM to determine which training state $s \in S_{train}$ $s \in S_{t r ain}$ is behaviorally equivalent under the inferred intent.
- If an object in the test state fulfills the same functional role as a training object (e.g., molding clay $\approx$ paintbrush under "pack art supplies"), it is mapped to that training object.
- Irrelevant objects are mapped to a "distractor" class.
Reward Evaluation: The learned reward function $R_\theta$ $R_{θ}$ (trained on $S_{train}$ $S_{t r ain}$ ) is applied to the aligned test trajectory. The alignment operator effectively projects novel states into the training domain along intent-relevant dimensions.
- Formula: $\tilde{R}_\theta(\xi' | \hat{\omega}) \triangleq R_\theta(f_{\hat{\omega}}(\xi'))$ .

3. Key Contributions

Intent-Conditioned Generalization: The paper proposes a paradigm shift from surface-level similarity (visual/linguistic) to intent-conditioned similarity. It demonstrates that high-level intent is the true driver of state equivalence in human-robot tasks.
Test-Time Adaptation without Retraining: GIFT enables a robot to generalize to unseen objects and layouts at test time by aligning states via an LLM, eliminating the need to re-collect data or retrain the reward model for every new environment.
Robustness to Confounding Factors: The framework explicitly handles "confounding" objects that share visual or linguistic features with training objects but differ functionally (e.g., a toothbrush vs. a paintbrush).
Real-World Validation: The method is validated not only in simulation but also on a physical 7-DoF Franka Panda robot, proving its viability in real-world settings.

4. Experimental Results

The authors evaluated GIFT on four tabletop manipulation tasks (Place Mug, Sweep Spill, Pack Backpack, Store Into Drawer) using over 50 unseen objects.

Baselines: Compared against Visual Similarity (DINO), Language Similarity (BERT), and an LLM without intent conditioning ( $LM_{No Intent}$ ).
Pairwise Win Rate: GIFT consistently achieved the highest win rate in predicting human preferences across all tasks.
- In complex tasks (Pack Backpack, Store Into Drawer), GIFT showed a ~20% improvement over the next best baseline.
- In simpler tasks, it showed a ~7% improvement.
State Alignment Accuracy (F1 Score): GIFT achieved superior F1 scores in aligning test objects to correct training equivalents.
- Failure Modes: Visual and language baselines suffered high False Positive rates (e.g., mapping a toothbrush to a paintbrush due to visual/lexical similarity) and False Negative rates (failing to map relevant art supplies like clay).
- Confounding Sets: On confounding subsets (objects with similar names or appearances but different functions), GIFT maintained low error rates, whereas baselines failed significantly.
Real-World Performance: Experiments on the Franka robot confirmed that GIFT could infer intent from physical demonstrations and successfully plan behaviors (e.g., packing clay, storing iPhones) that aligned with human preferences, while baselines failed due to spurious correlations.

5. Significance and Conclusion

Bridging the Sim-to-Real Gap: GIFT demonstrates that LLMs can effectively bridge the gap between limited training data and diverse real-world deployment scenarios by leveraging commonsense reasoning.
Solving Reward Misspecification: By focusing on intent, GIFT mitigates the problem of reward misspecification where robots optimize for proxies (like "brush-ness") rather than the true goal (like "art supplies").
Future Directions: The authors note limitations regarding the reliability of LLM inference (hallucinations, abstraction levels) and the need for uncertainty-aware intent kernels. However, the work establishes a strong foundation for flexible, intent-driven robot learning that adapts to distribution shifts without retraining.

In summary, GIFT provides a robust mechanism for robots to understand why a human performed a task, allowing them to apply that understanding to entirely new objects and environments, thereby significantly improving the reliability of autonomous agents in dynamic settings.