🤖 machine learning

Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach

This paper proposes a neurosymbolic imitation learning framework that combines neural networks' ability to process high-dimensional data with symbolic methods' generalization capabilities, leveraging privileged training information such as gaze data to improve efficiency and reduce overfitting.

Original authors: Nikhilesh Prabhakar, Varun Balaji, Athresh Karanam, Kristian Kersting, Sriraam Natarajan

Published 2026-05-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikhilesh Prabhakar, Varun Balaji, Athresh Karanam, Kristian Kersting, Sriraam Natarajan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to play a video game. You have two main ways to do this, but both have a major flaw.

The Problem: Two Flawed Teachers

The "Deep Learning" Teacher: This teacher is like a genius who has watched millions of hours of gameplay. They are great at seeing the screen and reacting instantly. However, they are a "black box." You can't ask them why they made a move, and they are terrible at generalizing. If you put a new enemy on the screen that they haven't seen before, they often freeze or panic. They also need a massive amount of data to learn, like a student who needs to read every book in the library before understanding a single concept.
The "Symbolic" Teacher: This teacher is like a logic professor. They learn clear, explainable rules (e.g., "If an enemy is on the left, move right"). They are great at explaining their moves and can handle new situations easily. But, they are terrible at looking at a raw video screen. They can't understand pixels; they need the world to be described to them in perfect, pre-written code.

The Solution: A Hybrid Tutor with a "Magic Gaze"
The authors of this paper, GRAIL, propose a new way to teach the robot that combines the best of both teachers. They call it Neurosymbolic Imitation Learning.

Think of it as a team of two:

The Eyes (Neural Part): A neural network that looks at the raw video screen and translates the pixels into a list of logical facts (e.g., "There is a player," "There is an enemy," "The enemy is to the left").
The Brain (Symbolic Part): A logic engine that takes those facts and applies rules to decide what to do.

The Secret Sauce: Privileged Information (The "Gaze")
Here is the clever twist. The authors realized that when humans play these games, their eyes don't look everywhere. They focus on the important things (like an enemy about to shoot) and ignore the background noise.

In the real world, we can't always see what a human is looking at while they play. But, for this experiment, the researchers had access to eye-tracking data (a heatmap showing exactly where the human was looking) only during the training phase.

They treated this eye-tracking data as "Privileged Information."

During Training: The robot gets to see the game screen and the human's eye movements. It learns: "Ah, the human is looking at the enemy, so that enemy is important. The human is ignoring the clouds in the sky, so I should ignore them too."
During Testing (The Real Game): The robot no longer sees the eye-tracking data. It only sees the screen. But because it learned which things to pay attention to during training, it now ignores the background noise on its own.

How It Works in Practice
Imagine the robot is playing a game like Seaquest (where you swim and shoot).

The Eyes: The robot looks at the screen and sees 50 objects. It creates a list of 50 facts.
The Gaze Filter: The robot remembers, "In training, the human only looked at the enemies and divers, not the bubbles." So, it uses that memory to "turn down the volume" on the bubbles and "turn up the volume" on the enemies.
The Logic Brain: Now, with a clean, focused list of important facts, the logic brain applies its rules: "If an enemy is close, fire."
The Result: The robot makes a move.

Why This is a Big Deal
The paper tested this on Atari games (like Asterix and Seaquest) and found three major wins:

Better Performance: The robot played better than the "Deep Learning" teacher and the "Symbolic" teacher alone.
Sample Efficiency: It learned much faster. While other robots needed to play thousands of times to get good, this robot got good with very few tries because the "gaze" helped it focus on what mattered immediately.
Generalization: When the game changed (e.g., suddenly there were 3 enemies instead of 1), the robot didn't panic. Because it learned relationships (e.g., "enemy is to the left") rather than just memorizing pixel patterns, it could handle new, unseen situations easily.

In a Nutshell
The paper presents a system that teaches a robot to play games by combining a "visual translator" with a "logical rule-maker." The secret weapon is using human eye-tracking data as a temporary training guide to teach the robot what to pay attention to. Once trained, the robot can play expertly on its own, even in new situations, without needing the human's eyes anymore. It's like teaching a student to study by showing them exactly which pages of the textbook are most important, so they learn to study effectively even when you aren't there to point at the pages.

Technical Summary: Neurosymbolic Imitation Learning with Human Guidance: A Privileged Information Approach

Problem Statement

Imitation Learning (IL) aims to learn a policy (a mapping from states to actions) from expert demonstrations. While pure neural methods (e.g., Behavioral Cloning) effectively handle high-dimensional raw data like images, they suffer from sample inefficiency, overfitting, and a lack of generalization to environments with varying numbers of objects. Conversely, pure symbolic methods offer interpretability and strong generalization but struggle to process high-dimensional inputs directly and often require pre-specified symbolic representations.

Furthermore, existing IL methods often fail to utilize Privileged Information (PI)—auxiliary data available during training but unavailable during deployment. While gaze data (human eye-tracking) is a natural candidate for PI in visual tasks, previous approaches typically apply gaze guidance to neural policies, preserving their "black-box" nature, or use it merely for regularization without altering the underlying symbolic reasoning process.

The core challenge addressed by this paper is: How can we develop an imitation learning framework that combines the high-dimensional processing of neural networks with the generalization and interpretability of symbolic reasoning, while effectively leveraging privileged information (specifically human gaze) to guide the learning of robust, generalizable policies?

Methodology: The GRAIL Framework

The authors propose GRAIL (Gaze-guided Relational Imitation Learning), a neurosymbolic framework that integrates three tightly coupled modules to learn interpretable probabilistic logic rules from raw images and gaze data.

1. Neurosymbolic Grounding (Perception Module)

Instead of assuming symbolic inputs, GRAIL accepts raw pixel observations ( $o_t$ ). A neural perception network ( $f_\theta$ ) maps these raw images to a set of probabilistic grounded atoms ( $v^{(0)}_t$ ).

Mechanism: The network outputs a vector of truth probabilities for a predefined set of first-order predicates (e.g., visibleEnemy, closeToPlayer).
Training: This module is trained independently using ground-truth binary labels derived from an object extraction oracle (OCAtari) via Negative Log-Likelihood (NLL) loss. Once trained, it acts as a fixed feature extractor.

2. Privileged Gaze Modulation

This is the core innovation leveraging the Learning Using Privileged Information (LUPI) paradigm.

Gaze Prediction: A separate network ( $g_\phi$ ) is trained to predict a gaze heatmap from the raw frame, minimizing Kullback-Leibler divergence against ground-truth eye-tracking data.
Atom Reweighting: During policy training, the gaze heatmap is used to compute a "gaze mass" ( $s_i$ ) for each entity referenced in the grounded atoms.
Modulation: The initial atom probabilities are reweighted by the gaze scores: $v^{(g)}_{t,i} = v^{(0)}_{t,i} \cdot s_i$ .
Effect: This mechanism suppresses the probabilities of atoms corresponding to background or task-irrelevant entities that the human expert did not look at, effectively filtering out spurious correlations before the logical reasoning stage.

3. Differentiable Forward-Chaining Reasoner (Policy Learning)

The gaze-modulated symbolic state ( $v^{(g)}_t$ ) is fed into a Neurosymbolic Forward Reasoner (NSFR).

Logic: The reasoner performs differentiable forward-chaining inference over a predefined library of First-Order Logic (FOL) definite clauses (rules).
Differentiability: Logical operations (conjunction/disjunction) are approximated using fuzzy logic semantics (product T-norm and T-conorm), allowing gradients to flow through the inference graph.
Policy Optimization: The system learns a weight matrix ( $W$ ) for the rules. The objective is Behavioral Cloning: minimizing the negative log-likelihood of the expert's action given the inferred action distribution derived from the rule activations.

Key Contributions

Novel Neurosymbolic Framework: The development of GRAIL, which learns interpretable probabilistic logic rules for imitation learning directly from raw visual data.
Privileged Information Integration: The specific formulation of human gaze as privileged information to modulate symbolic atom valuations. This is distinct from prior work that applies gaze to neural layers; here, gaze guides the symbolic grounding stage.
Interdisciplinary Synthesis: The paper combines concepts from neural learning, symbolic learning (Inductive Logic Programming), imitation learning, and advice-based learning (LUPI) into a single end-to-end trainable system.
Empirical Validation: Demonstration that the framework is sample-efficient, learns robust policies, and generalizes to test-time scenarios with an unknown number of objects, outperforming both purely neural and purely symbolic baselines.

Experimental Results

The framework was evaluated on two Atari 2600 environments (Asterix and Seaquest) using the Atari-HEAD dataset (which includes human eye-tracking data).

Effectiveness (Q1): GRAIL achieved the highest mean game scores in both domains, significantly outperforming purely neural baselines (Behavioral Cloning, AGIL, BC+Mask) and the neurosymbolic baseline without gaze modulation (NSFR-IL).
Sample Efficiency (Q2): GRAIL converged to higher performance levels with significantly fewer training samples. Notably, GRAIL trained on only 10% of the data outperformed the NSFR-IL baseline trained on the full dataset. The authors attribute this to gaze modulation acting as a regularizer, preventing the model from fitting spurious correlations in the data.
Generalization (Q3): When tested on environments with a higher number of objects than seen during training, GRAIL maintained high performance. Neural baselines failed to generalize beyond their training object counts, while the relational nature of GRAIL's symbolic representation allowed it to handle unseen object counts naturally.

Significance and Claims

The paper claims that GRAIL represents the first end-to-end neurosymbolic framework capable of seamlessly integrating gaze information as privileged information to learn effective, efficient, and generalizable policies.

Robustness: By using gaze to filter symbolic atoms, the system avoids learning policies based on irrelevant background features, addressing the "covariate shift" problem common in imitation learning.
Interpretability: Unlike neural policies, the resulting policy is a set of weighted logic rules. The weights themselves provide an explanation of the decision-making process (e.g., which rules are most critical for specific actions).
Modularity: The framework is described as "plug-and-play," capable of integrating differentiable learners, various perception modules, and different forms of privileged information (not limited to gaze).

The authors conclude that combining classical AI concepts (neural, symbolic, advice-based) yields a model that is not only performant but also transparent and capable of generalizing to complex, dynamic environments where traditional symbolic methods struggle and pure neural methods lack robustness.