🤖 AI

Offline Discovery of Interpretable Skills from Multi-Task Trajectories

This paper introduces LOKI, a three-stage offline learning framework that discovers interpretable, reusable skills from multi-task demonstrations without explicit rewards, enabling the construction of hierarchical policies that achieve high success rates on complex benchmarks and can compose learned skills to solve novel tasks.

Original authors: Chongyu Zhu, Mithun Vanniasinghe, Jiayu Chen, Chi-Guhn Lee

Published 2026-02-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chongyu Zhu, Mithun Vanniasinghe, Jiayu Chen, Chi-Guhn Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you want to teach a robot how to cook a complex meal, like a full dinner. You don't have time to write a line-by-line manual for every single movement (like "move arm 2 inches left, then rotate wrist 15 degrees"). Instead, you show the robot a video of a human chef doing the whole job.

The problem is, the video is just one long, continuous stream of motion. The robot doesn't know where one action ends and the next begins. It doesn't know that "chopping vegetables" is one distinct skill, and "turning on the stove" is another. It just sees a blur of movement.

This paper introduces a new method called LOKI (Learning Offline Key-skills via Imitation) to solve this. Think of LOKI as a smart editor that watches hours of raw cooking videos and automatically cuts them into perfect, reusable "clips" or "skills" that the robot can learn and mix and match later.

Here is how LOKI works, broken down into three simple stages:

Stage 1: The "Big Picture" Editor (Macro-Segmentation)

First, LOKI looks at the whole video and tries to find the major chapters.

The Analogy: Imagine you have a long movie, but you don't know where the scenes change. LOKI uses a special tool (called an EVQ-VAE) that acts like a "task detective."
How it works: It knows the general goal of the video (e.g., "Make Dinner"). It looks for moments where the robot is doing something unique to that specific task (like opening a specific microwave) versus something generic that happens in many tasks (like moving an arm from point A to point B).
The Result: It cuts the long video into big chunks called General Skills. It's like saying, "Okay, this whole section is about the microwave, and this next section is about the stove."

Stage 2: The "Fine-Tuning" Editor (Micro-Segmentation)

Now, LOKI takes those big chunks and looks closer.

The Analogy: Even within the "Microwave" section, the robot might be doing three different things: walking to the microwave, grabbing the handle, and turning the dial. The first stage was too broad; it needs to find the exact moment one action stops and the next begins.
How it works: LOKI uses a "reconstruction" trick. It tries to predict what the robot will do next based on what it just did. If the robot suddenly does something unexpected or the prediction fails, LOKI knows, "Ah! The robot just switched to a new skill!" It then uses a clustering process (grouping similar movements together) to clean up the cuts, removing messy or unnecessary boundaries.
The Result: It creates Independent Skills. These are the atomic, reusable moves, like "Turn Knob" or "Open Door," that are consistent across different tasks.

Stage 3: The "Director" (Hierarchical Policy)

Finally, LOKI teaches the robot how to use these clips.

The Analogy: Imagine a movie director. The director doesn't tell the actor how to blink or breathe; they just say, "Now, do the 'Turn Knob' clip."
How it works:
1. The Low-Level Actor: The robot learns to perform each specific "clip" perfectly.
2. The Stop Signal: LOKI teaches the robot a special signal (a "termination function") that says, "You are done with this clip; stop now." This prevents the robot from trying to turn the knob forever.
3. The Script: The robot is given a simple script (a sequence of skill IDs) to follow. It just needs to know: "Do Clip A, then Clip B, then Clip C."

Why is this a big deal?

No Manual Labels Needed: Usually, to teach a robot, humans have to manually label every single step ("Now he opens the fridge"). LOKI figures this out almost automatically, needing only a tiny hint about the overall goal.
It Makes Sense to Humans: The skills LOKI discovers aren't random math patterns; they align with how humans think. If you look at the robot's "brain," you can see it groups "opening a cabinet" and "opening a microwave" together because they feel similar, just like a human would.
Mix and Match: Because the skills are broken down so cleanly, the robot can take a skill it learned for one task (like "turning a light switch") and use it in a completely new, unseen task.

In summary: LOKI takes messy, unlabelled videos of robots doing complex jobs, automatically chops them into logical, human-understandable "skills," and teaches the robot how to string those skills together to get the job done. It turns a chaotic stream of data into a clean, organized instruction manual.

Technical Summary: Offline Discovery of Interpretable Skills from Multi-Task Trajectories (LOKI)

Problem Statement

Hierarchical Imitation Learning (HIL) offers a promising paradigm for acquiring complex robot behaviors by decomposing long-horizon tasks into manageable subtasks. However, a critical bottleneck remains in discovering reusable, semantically meaningful skills from offline, multi-task trajectory data that lacks explicit rewards or subtask annotations. Existing methods face three primary challenges:

Ambiguity in Skill Boundaries: Accurately segmenting continuous trajectories into discrete skills without explicit supervision is difficult.
Lack of Reusability and Interpretability: Learned skills often fail to align with human conceptualizations or transfer across tasks.
Scalability: Many approaches are tailored to single-task settings or require extensive task-specific engineering, limiting their utility in diverse, multi-task environments.

The paper proposes LOKI (Learning Offline Key-skills via Imitation), a framework designed to address these gaps by discovering robust, interpretable skills directly from raw, unannotated offline demonstrations.

Methodology: The LOKI Framework

LOKI is a three-stage, end-to-end learning framework that transforms raw multi-task trajectories into an executable hierarchical policy.

Stage 1: Macro-Segmentation via Enforced VQ-VAE (EVQ-VAE)

The first stage performs coarse, task-aware segmentation to distinguish between General Skills (GS).

Architecture: The authors introduce an Enforced Vector Quantized Variational Autoencoder (EVQ-VAE). Unlike standard VQ-VAEs, this architecture leverages weak task labels ( $c$ ) to enforce task-conditional quantization.
Loss Functions: The model minimizes a composite loss including task-conditional reconstruction, codebook, and commitment losses. Crucially, it introduces a divergence loss ( $L_{divergence}$ ) to maximize the Euclidean distance between codebook vectors, preventing collapse and encouraging distinct representations.
Boundary Detection: Skill boundaries are identified using latent space entropy.
- Extrinsic skills (task-specific) result in low entropy as embeddings align closely with a single task-specific codebook vector.
- Intrinsic skills (task-agnostic) result in high entropy as embeddings are equidistant to multiple vectors.
- Change points are detected using the Pelt algorithm on the entropy signal, separating intrinsic and extrinsic behaviors.

Stage 2: Weakly-Supervised Micro-Segmentation

This stage refines the macro-segments into fine-grained, context-independent Independent Skills (IS).

Sequential Modeling: A sequential VAE processes fixed-length sliding windows of the macro-segments. The model is trained to reconstruct actions and states while regularizing the latent space against a prior conditioned only on the initial state and task label (ensuring the skill code does not "cheat" by observing future states).
Boundary Refinement:
1. Error Spike Detection: Potential boundaries are identified by detecting sudden spikes in reconstruction error (indicating a deviation from current skill dynamics).
2. Iterative Clustering: A K-Means clustering algorithm is applied to segment embeddings. Consecutive segments assigned to the same cluster are merged, removing redundant boundaries.
Task Skill Alignment: The framework distinguishes between reusable (intrinsic) and unique (extrinsic) skills. Intrinsic segments are clustered into a shared set of primitives, while extrinsic skills receive unique IDs. Finally, all trajectories for a specific task are force-aligned to a canonical skill sequence based on frequency and similarity, ensuring structural consistency.

Stage 3: Hierarchical Policy Construction

Using the refined skill boundaries, the framework constructs an executable policy:

Termination Function ( $\beta$ ): A Multi-Layer Perceptron is trained as a binary classifier to predict skill termination based on the current state, replacing fixed time horizons with state-based switching.
Low-Level Policy: A skill-conditioned controller ( $\pi(a_t|s_t, k)$ ) is trained via conditional behavior cloning to execute specific atomic skills.
High-Level Policy: Instead of learning a dynamic meta-controller, the system utilizes the canonical skill sequences derived in Stage 2 as deterministic task programs.

Key Contributions

LOKI Framework: A novel three-stage, weakly-supervised framework for offline skill discovery and hierarchical imitation from long-horizon, multi-task data.
EVQ-VAE: A new architecture that utilizes weak task labels to enforce task-conditional quantization, enabling the separation of intrinsic and extrinsic skills.
Micro-Segmentation Method: A weakly-supervised approach combining reconstruction error analysis and iterative clustering to distinguish conservative, granular skills.
Empirical Validation: Demonstration that discovered skills are semantically meaningful, consistent, and composable, enabling the solution of novel tasks.

Experimental Results

Experiments were conducted on the D4RL Kitchen benchmark, utilizing the kitchen-mixed-v0 dataset containing trajectories for seven subtasks (e.g., microwave, kettle, light switch) combined into complex multi-stage tasks.

Macro-Segmentation: The EVQ-VAE successfully identified boundaries between intrinsic and extrinsic skills, evidenced by distinct low-entropy (extrinsic) and high-entropy (intrinsic) states in the latent space.
Micro-Segmentation: The iterative refinement process effectively removed noisy boundaries and enforced consistent segmentation structures across trajectories of the same task.
Interpretability: PCA visualization of the latent space revealed that skills with similar robot motions (e.g., "Microwave" and "Hinge Cabinet") clustered adjacently, confirming semantic alignment with human intuition.
Termination Accuracy: The learned termination function achieved 93.7% accuracy within a $\pm4$ timestep window of ground-truth boundaries.
Performance Comparison:
- LOKI variants (LOKI-O using one-hot IDs and LOKI-E using continuous embeddings) significantly outperformed hierarchical baselines DADS and OPAL in terms of low-level policy Mean Squared Error (MSE).
- LOKI-E achieved the best performance among the proposed methods (Average MSE: 7.76), attributed to the richer information in continuous embeddings compared to discrete IDs.
- While the "Expert" (flat BC) baseline achieved lower MSE, LOKI demonstrated superior modularity and interpretability, which are critical for hierarchical control.

Significance and Claims

The paper claims that LOKI provides a practical and robust method for extracting structured, interpretable skills from raw offline data without requiring dense rewards or manual subtask annotations. By aligning discovered skills with human intuition and enabling their composition to solve novel tasks, the work addresses a fundamental hurdle in scaling imitation learning to complex, multi-task robotic environments. The authors position this as a significant step toward more scalable and generalizable robot learning from demonstrations, offering a viable alternative to methods that rely on extensive online interaction or massive, annotated datasets.