💬 NLP

Latent Preference Modeling for Cross-Session Personalized Tool Calling

This paper introduces the MPT benchmark and the PRefine method to address under-specified user requests in tool-augmented agents by modeling evolving user preferences through a memory-augmented generate-verify-refine loop, achieving high accuracy with significantly reduced token usage.

Original authors: Yejin Yoon, Minseo Kim, Taeuk Kim

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yejin Yoon, Minseo Kim, Taeuk Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Forgetful" Assistant

Imagine you have a super-smart robot assistant. You ask it to book a flight, and it asks, "What class do you want? Economy or First?" You say, "Economy."

The next day, you ask it to find a restaurant. It asks, "Do you want something fancy or cheap?" You say, "Cheap."

The day after that, you ask it to rent a car. It asks, "Do you want a luxury SUV or a compact car?" You say, "Compact."

Now, imagine you come back a week later and say, "Book me a flight for my trip."

A standard AI might just say, "Okay, what class do you want?" because it only looks at this specific conversation. It has forgotten that you have a consistent pattern: you always choose the budget-friendly option.

The problem is that users often forget to mention these details every single time. They expect the AI to remember their "vibe" or "style" across different days and different tasks. But current AIs are like goldfish; they forget everything once the conversation ends.

The Solution: Learning the "Why," Not Just the "What"

The researchers (Yejin Yoon, Minseo Kim, and Taeuk Kim) realized that to fix this, the AI needs to stop just memorizing what you did (e.g., "User picked Economy") and start understanding why you did it (e.g., "User is budget-conscious").

They call this Latent Preference Modeling.

Latent means "hidden." It's the invisible rule driving your choices.
Preference is your personal style.

Think of it like a tailor.

A bad tailor measures you for a suit, then forgets you. Next time you come in, they ask, "How big are you?"
A great tailor remembers that you are "slim but tall" and "prefer dark colors." Even if you don't say it, they cut the fabric that way automatically.

The New Benchmark: MPT (The "Exam" for AIs)

To test if AIs can do this, the team created a new test called MPT (Multi-Session Personalized Tool Calling). It's like a final exam for AI assistants with three levels of difficulty:

Preference Recall (The Easy Level): The AI sees you booked an Economy flight yesterday. Today, you ask for a flight again. The AI just copies yesterday's answer. Easy.
Preference Induction (The Detective Level): You booked a cheap flight, a cheap hotel, and a cheap rental car on three different days. Today, you ask for a flight again, but the AI has never seen you book a flight before. It has to look at your hotel and car choices and say, "Ah, this person is budget-conscious, so I'll pick Economy." Harder.
Preference Transfer (The Genius Level): You have a history of buying "eco-friendly" products (bottles, bags, clothes). Today, you ask to book a flight. The AI has never seen you buy eco-friendly flights, but it realizes, "This person cares about the environment," and picks a flight with a carbon offset. Very Hard.

The New Method: PREFINE (The "Smart Notebook")

The team also built a new method called PREFINE to help AIs pass this exam.

Instead of feeding the AI the entire history of your life (which is huge, messy, and confusing), PREFINE acts like a smart, evolving notebook.

Here is how it works, using a "Generate-Verify-Refine" loop:

Generate (The Guess): The AI looks at your recent actions and makes a guess about your hidden rule.
- AI thinks: "Maybe the user likes red cars?"
Verify (The Fact-Checker): A second part of the AI acts like a strict editor. It checks the notebook.
- Editor says: "Wait, you only bought a red car once. That's not a rule. That's just a one-time thing. Reject this guess."
Refine (The Correction): The AI tries again with a better guess.
- AI thinks: "Okay, they bought a red car, a red bike, and a red phone. They like red."
- Editor says: "Yes! That is a consistent pattern. Write that down in the notebook."

The Magic:
Over time, this notebook gets very small and very powerful. Instead of remembering 10,000 chat messages, the AI only keeps a few sentences like: "User prefers budget options" or "User likes compact sizes."

Why This Matters

The paper found two amazing things:

It's Super Efficient: The old way of doing this (reading the whole history) uses a massive amount of computer power. PREFINE uses only 1.24% of the energy and memory. It's like reading a 500-page book vs. reading a 5-page summary that contains all the important plot points.
It Actually Works: When the AI uses this "Smart Notebook," it gets much better at guessing what you want, even when you don't say it. It handles the "Detective" and "Genius" levels of the exam much better than other methods.

The Takeaway

The future of AI assistants isn't about remembering everything you ever said. It's about remembering who you are.

By turning your messy history into a few clear, hidden rules (like "I'm a budget shopper" or "I love spicy food"), AI can finally stop asking, "What do you want?" and start saying, "I know what you want. Here it is."

1. Problem Definition

The paper addresses a critical limitation in current LLM-based agents: under-specified tool calls. Users frequently omit essential details in their requests (e.g., "Book a flight" without specifying class), forcing agents to rely on incomplete inputs. While agents can retrieve explicit past actions, they struggle to infer latent preferences—implicit, persistent constraints on decision-making derived from recurring behavioral patterns across multiple, unordered, and noisy dialogue sessions.

The authors argue that personalized tool calling is not merely a memory retrieval problem but a reasoning problem. Existing benchmarks and methods often assume preferences are explicitly provided (via profiles) or rely on direct pattern matching within a single domain. They fail to address the challenge of:

Preference Induction: Aggregating evidence across different domains to infer a latent constraint.
Preference Transfer: Applying a preference learned in one domain (e.g., budget-consciousness in hotels) to a new, unseen domain (e.g., flight booking).

2. Methodology

A. Benchmark: MPT (Multi-Session Personalized Tool Calling)

To evaluate this problem, the authors introduce MPT, a benchmark built upon the Schema-Guided Dialogue (SGD) dataset.

Structure: It consists of 265 multi-session dialogues (2,020 sessions, ~40k turns) where a single user interacts with an agent over time.
Task: The agent must predict missing API arguments for a current query by reasoning over the accumulated interaction history ( $S, A_{\le T}$ ).
Three Challenge Types:
1. Preference Recall: Direct reuse of a preference within the same domain (e.g., always choosing "Economy" class for flights).
2. Preference Induction: Inferring a latent preference by aggregating cross-domain evidence (e.g., choosing "cheap" restaurants, "free" attractions, and "compact" cars to infer a "budget-conscious" preference).
3. Preference Transfer: Applying an inferred preference from a source domain to a target domain where no direct evidence exists.
Annotation: Preferences are annotated at a high level (e.g., "Budget," "Travel Style") rather than specific slot values, ensuring the taxonomy generalizes across different API schemas.

B. Proposed Method: PREFINE

The authors propose PREFINE, a test-time memory-augmented method that treats latent preferences as revisable hypotheses rather than static facts. It operates via a Generate–Verify–Refine loop:

Generator: At each new session, the generator proposes candidate preference hypotheses ( $h$ ) based on the current dialogue, the executed tool call, and the previous memory state. It aims to abstract specific actions into general constraints (e.g., "User prefers budget-friendly options").
Verifier: A verifier module evaluates each hypothesis against four criteria:
- Evidence Support: Is the claim backed by multiple consistent interactions?
- Abstraction Quality: Is it a generalizable constraint or just a restatement of a specific slot?
- Actionability: Can it actually constrain future API arguments?
- Temporal Consistency: Does it align with the most recent stable behavior?
Refine: If a hypothesis fails verification (e.g., it is too specific or unsupported), it is returned to the generator for refinement. Only verified, abstract constraints are stored in the memory ( $M_T$ ).
Schema-Agnostic Memory: The memory stores abstract constraints (textual hypotheses) rather than schema-specific API signatures. This allows the system to ground these preferences into new or evolving API schemas at inference time.

3. Key Contributions

MPT Benchmark: The first benchmark specifically designed to evaluate cross-session latent preference modeling, distinguishing between simple recall, cross-domain induction, and transfer.
PREFINE Framework: A novel, lightweight method that models preferences as evolving hypotheses. Unlike RAG or static memory systems, it actively filters out noise and over-specificity through iterative verification.
Empirical Insights: Demonstration that robust personalization requires capturing the reasons behind user choices (latent constraints) rather than just the choices themselves.

4. Experimental Results

The authors evaluated PREFINE against baselines including Full-Dialogue Prompting, RAG, Mem0, and LangMem across various LLM backbones (Gemma, GPT-4o/5, Llama, etc.).

Performance Gains:
- PREFINE significantly outperforms baselines in Preference Induction and Preference Transfer, where baselines often fail (e.g., F1 drops from ~53% in Recall to ~16% in Transfer for base prompting).
- PREFINE improves Preference Exact Match (P-EM) by an average of 13.11% in context-guided queries and 14.81% in context-free queries.
- It also improves Explicit-Argument F1 (EA-F1), suggesting that providing compact, verified preferences helps models focus on the current query rather than getting lost in long histories.
Efficiency:
- PREFINE uses only 1.24% of the tokens required by full-history prompting (averaging ~23 tokens per dialogue vs. thousands for full history).
- Memory footprint remains constant (~20–25 tokens) even as sessions accumulate, unlike other methods where memory grows linearly.
Dynamic Schema Robustness:
- In a dynamic schema evaluation (testing on unseen API domains with different slot names), PREFINE maintained high performance (e.g., P-EM rose from 3.75% to 47.00% with GPT-5), whereas RAG and LangMem failed due to lexical mismatches. This confirms the schema-agnostic nature of PREFINE's memory.

5. Significance

This work shifts the paradigm of agent personalization from retrieval to reasoning. It demonstrates that:

Abstraction is key: Agents must abstract behavioral patterns into reusable constraints to handle long-horizon, multi-session interactions.
Memory quality > Memory quantity: Storing a few verified, abstract hypotheses is more effective than storing vast amounts of raw dialogue history.
Generalization: By decoupling memory content from specific API schemas, agents can adapt to evolving tool interfaces without retraining or re-annotation.

The paper concludes that future agentic systems must incorporate mechanisms to infer and update "latent user constraints" to handle the ambiguity and incompleteness inherent in real-world human-agent interactions.