💬 NLP

$How^{2}$ : How to learn from procedural How-to questions

The paper introduces $How^{2}$ , a memory agent framework that enables LLM-based agents to improve lifelong planning in interactive environments by asking and storing procedural how-to questions, demonstrating that abstract, state-independent answers are most effective for learning.

Original authors: Gautier Dagan, Frank Keller, Alex Lascarides

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gautier Dagan, Frank Keller, Alex Lascarides

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a complex piece of furniture, like a bookshelf, but you've never done it before. You have a pile of wood and tools, but no instructions.

In the world of Artificial Intelligence, this is what an "agent" (a smart computer program) faces when it tries to solve a planning problem. It knows what it wants to build, but it doesn't know how to put the pieces together.

This paper, titled How2, introduces a new way for these AI agents to learn. Instead of just guessing and making mistakes (trial and error), the AI is taught to ask a "Teacher" (another AI or a human expert) for help, write down the answer, and use that note to solve similar problems later.

Here is the breakdown of their discovery, using simple analogies:

1. The Problem: The "Too Specific" vs. "Too Vague" Dilemma

When you ask a teacher "How do I build this bookshelf?", they can answer in different ways. The researchers tested four types of answers to see which one helps the student learn best over time:

The "GPS" Answer (Executable): The teacher says, "Move your hand exactly 3 inches to the left, then pick up the screw in slot #12."
- Pros: It works perfectly right now.
- Cons: If you move the screw to slot #15, the instructions are useless. It's like a GPS that only works if you are in the exact same traffic jam as before.
The "Sub-Goal" Answer (Partially Executable): The teacher says, "First, find a screw. Then, put it in the hole."
- Pros: It's more flexible. You can find the screw wherever it is.
The "Abstract" Answer (Non-Executable): The teacher says, "You need to arrange the wood in a 'T' shape and screw it together."
- Pros: This is the most flexible. It doesn't care about specific slots or numbers; it describes the pattern.
- Cons: The AI has to figure out exactly which pieces fit the "T" shape.

2. The Big Discovery: Short-term vs. Long-term

The researchers found a fascinating trade-off, like choosing between a takeout meal and learning to cook:

For immediate success: The "GPS" answer (specific, step-by-step instructions) is the best. If you just need to build the bookshelf right now, follow the exact steps.
For lifelong learning: The "Abstract" or "Sub-Goal" answers are much better. If you want to build many bookshelves over your life, you need to understand the concept of the pattern, not just the specific coordinates of one screw.

The Analogy:
If a teacher gives you a specific list of ingredients for a cake (e.g., "Use the flour in the blue bag on the top shelf"), you can make that one cake. But if the blue bag is empty tomorrow, you're stuck.
If the teacher says, "Use two cups of flour," you can make the cake no matter where the flour is or what color the bag is. The paper shows that AI agents learn much better when they are taught the "two cups of flour" rule rather than the "blue bag" rule.

3. The Solution: The "How2" Framework

The authors built a system called How2 to manage this learning process. Think of it as a smart notebook for the AI.

Here is how it works in four steps:

Check the Notebook: Before trying to build something, the AI checks its memory. "Have I built a bookshelf like this before?"
Ask the Teacher: If the notebook is empty or the old notes don't fit the current situation (e.g., the wood is in a different spot), the AI asks the Teacher, "How do I do this?"
Translate the Answer: This is the magic step. When the Teacher gives an answer, the AI doesn't just copy-paste it. It translates the answer.
- Example: If the teacher says "Move the wood from Slot 12," the AI's notebook rewrites it as "Move the wood from wherever it is." This makes the note useful for any future situation, not just the current one.
Store and Reuse: The AI saves this "translated" note. Next time it needs to build a bookshelf, it reads the note, figures out where the wood is now, and follows the general rule.

4. The Results: The "Minecraft" Test

The researchers tested this in a digital world called Plancraft (based on the game Minecraft), where the AI has to craft items like glass bottles or red dye using a crafting grid.

The Finding: Agents that just followed specific, rigid instructions (the "GPS" style) failed miserably when the game setup changed slightly. They couldn't adapt.
The Winner: Agents that used the How2 system with "translated" notes (abstracting away the specific slot numbers) became much smarter over time. They asked for help less often and solved more tasks on their own because they had learned the patterns of crafting, not just the specific moves.

Summary

The paper argues that for AI to truly learn and get better at planning over a lifetime, it shouldn't just memorize specific instructions. Instead, it should ask questions, get answers, and then summarize those answers into general rules.

It's the difference between memorizing a single phone number (useless if the person moves) and understanding how a phone book works (useful forever). The How2 framework teaches AI to do exactly that: turn specific "how-to" answers into general, reusable knowledge.

Technical Summary: How2 – Learning from Procedural How-to Questions

Problem Statement

Agents operating in interactive environments often face planning problems where they lack the necessary procedural knowledge to complete specific tasks. While AI assistants can ask clarifying questions to resolve ambiguity, they rarely ask for procedural knowledge (e.g., "How do I X?") to fill knowledge gaps for future planning. The challenge lies in the open-ended nature of "how-to" questions: valid answers range from high-level sub-goal descriptions to fully executable action sequences. This variability makes it difficult for agents to ask the right questions and for "teachers" (human or oracle) to provide answers that are both immediately useful and reusable for lifelong learning. Specifically, answers tightly coupled to the current state (e.g., specific inventory slots) are often not generalizable to new states, hindering long-term autonomy.

Methodology

The authors propose How2, a memory-driven agent framework designed for lifelong learning in interactive environments. The system operates within Plancraft, a Minecraft-based crafting environment where agents must manipulate inventory items to assemble target objects.

Core Framework

How2 integrates an agent with a teacher and a memory module through the following workflow:

Memory Retrieval: The agent first queries a memory store using a read-memory tool with a query $\theta$ (e.g., a recipe name). The memory is a key-value store where values are sets of memory entries.
Relevance Check: If entries exist, an LLM-based relevance check determines if the stored memory applies to the current game state ( $o_t$ ).
Question Generation: If no relevant memory is found (a cache miss), the agent generates a procedural "how-to" question ( $q_\theta$ ) conditioned on the current observation.
Teacher Response: A teacher model ( $T$ $T$ ) answers the question. The paper evaluates four distinct teacher types based on the abstraction level of their responses:
- Executable: Provides fully grounded, step-by-step action sequences (e.g., "Move from I12 to A1").
- Partially-Executable: Removes state-specific details (e.g., "Move glass to A1") but remains a flat list of actions.
- Subgoal-Partially-Executable: Structures the plan into hierarchical subgoals (e.g., "Smelt glass" followed by "Craft bottle"), grouping actions into identifiable recipes.
- Non-Executable: Provides high-level, ungrounded natural language descriptions using patterns and shapes (e.g., "Arrange items in a V-shape"), abstracting away specific slot names entirely.
Parsing and Storage: The agent's ParseAnswer module processes the teacher's response. It abstracts state-specific details (replacing specific slot IDs with item names) and generates tags. The generalized entry is stored in memory under the original query and associated tags.

Experimental Setup

The authors evaluated the framework using the Llama 3.3 70B and Qwen 3 32B models. They introduced two dataset splits to test lifelong learning capabilities:

Low Repetition: The original Plancraft validation set with 347 unique tasks.
High Repetition: A new split with 107 unique tasks repeated across 570 examples, designed to test knowledge re-use.

The evaluation compared the full How2 framework against ablations:

Just Ask: An oracle setup where the teacher answers every query without memory storage (upper bound).
Memory-Only: Using memory without parsing or relevance checks.
Parse/Relevance: Individual components of the full framework.

Key Contributions

The How2 Framework: A novel architecture enabling agents to learn procedural knowledge by asking questions, storing answers, and reusing them via a memory module.
Analysis of Abstraction Levels: A systematic investigation into how the granularity of teacher answers (from executable to non-executable) impacts both immediate task success and long-term reusability.
Trade-off Identification: The identification of a critical trade-off between the immediate utility of an answer and its long-term reusability.

Results

The experiments yielded several key findings:

Immediate vs. Long-term Utility:
- Executable teachers achieved the highest success rates in the "Just Ask" setting (0.59), confirming they are most effective for immediate execution.
- However, when reused in the Memory-Only setting, executable plans suffered a dramatic performance drop (from 0.58 to 0.32 in the high-repetition split) due to their tight coupling with specific states.
- Abstracted answers (Subgoal-Partially-Executable and Non-Executable) showed significantly higher reusability. Their performance dropped by only ~9% when moving from "Just Ask" to "Memory" settings.
Effectiveness of Abstraction:
- The Subgoal-Partially-Executable teacher consistently outperformed the flat Partially-Executable teacher, supporting the hypothesis that structuring answers into subgoals enhances memory effectiveness.
- The Parse module was crucial for making executable plans reusable, improving their success rate in the high-repetition split from 0.32 to 0.44.
Full Framework Performance:
- The full How2 framework (integrating memory, parsing, and relevance checks) achieved a success rate of 0.52 in the high-repetition setting.
- Crucially, this was achieved with a 42% lower intervention rate (0.53) compared to the "Just Ask" oracle (0.92), demonstrating that the agent successfully reused stored knowledge rather than constantly querying the teacher.
- The framework allowed agents using Non-Executable teachers (which are hardest to ground) to achieve success rates (0.53) approaching those of agents with fully executable teachers, by effectively grounding abstract instructions through the parse and relevance modules.
Model Variations:
- Experiments with the Qwen 3 32B reasoning model showed similar trends, though the model utilized the memory action less frequently. Interestingly, the Non-Executable teacher performed best with Qwen 3, suggesting reasoning capabilities may better handle abstract, ungrounded instructions.

Significance and Claims

The paper claims that How2 provides a viable mechanism for LLM-based agents to improve planning capabilities over time in interactive environments without fine-tuning. The primary significance lies in demonstrating that abstracting knowledge from how-to questions is essential for effective lifelong learning.

While direct, executable instructions are optimal for solving a single instance of a task, they fail to generalize. Conversely, abstracted answers (subgoals or high-level patterns), when combined with a parsing and relevance-checking mechanism, allow agents to build a robust, reusable knowledge base. This reduces reliance on external supervision (teacher interventions) while maintaining high task success rates, effectively bridging the gap between immediate problem solving and long-term skill acquisition. The authors conclude that learning from procedural questions is a powerful strategy for agents operating in constrained, interactive domains.