💬 NLP

Shopping Companion: A Memory-Augmented LLM Agent for Real-World E-Commerce Tasks

This paper addresses the challenges of long-term preference-aware shopping by introducing a new benchmark and "Shopping Companion," a unified memory-augmented LLM agent trained with a dual-reward reinforcement learning strategy that outperforms state-of-the-art models in capturing user preferences and executing complex e-commerce tasks.

Original authors: Zijian Yu, Kejun Xiao, Huaipeng Zhao, Tao Luo, Xiaoyi Zeng

Published 2026-03-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zijian Yu, Kejun Xiao, Huaipeng Zhao, Tao Luo, Xiaoyi Zeng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a personal shopping assistant named Alex. You've known Alex for years. You've told Alex, "I hate red shoes," "I always buy size 43," and "I need running gear that dries quickly."

Now, you walk into a massive, warehouse-sized store with 1.2 million items on the shelves. You tell Alex, "I need a new running outfit for under $300, and I have a coupon."

The Problem:
Most current AI assistants are like amnesiacs. They might be great at finding a shoe right now, but they forget you hate red shoes or that you need size 43 unless you remind them every single time. Worse, if you try to buy a whole outfit (shoes + shorts) with a budget, they often get confused, buy the wrong size, or forget the coupon.

Existing tests for these AI assistants are like driving tests on empty parking lots. They don't test if the AI can remember your long-term habits while navigating a busy, complex city street.

The Solution: "Shopping Companion"
The authors of this paper built a new, super-challenging test and a new type of AI assistant called Shopping Companion. Here is how it works, using simple analogies:

1. The "Memory Vault" (Long-Term Memory)

Instead of just looking at what you said this morning, Shopping Companion has a Memory Vault.

How it works: Before it even starts shopping, it digs through your past conversations (like reading your old diary entries) to find clues about your size, your style, and what you dislike.
The Twist: It doesn't just guess. It shows you a summary: "I found you like size 43 and hate carbon plates. Is that right?" This lets you step in and say, "Actually, I changed my mind," or "Yes, that's correct." This is called User Intervention.

2. The Two-Step Dance (Two-Stage Framework)

Most AI tries to do everything at once, which is like trying to tie your shoes while running a marathon. Shopping Companion breaks it down:

Step 1: The Detective. It investigates your history to figure out exactly what you want.
Step 2: The Shopper. Once you confirm the list, it goes to the shelves, checks the prices, applies the coupon, and finds the perfect bundle.

3. The "Coach" (Reinforcement Learning)

How do you teach an AI to be this good? You can't just say "Good job" at the very end.

The Analogy: Imagine teaching a dog to fetch. If you only give it a treat when it brings the ball back, it might get confused about which part of the action was good (picking it up? running? dropping it?).
The Paper's Trick: The authors gave the AI a "Tool-Wise Reward." Every time the AI uses a tool correctly (like searching the memory vault or checking a product price), it gets a tiny "high-five" (a reward point). If it wastes time or picks the wrong tool, it gets a gentle "no." This teaches the AI to be efficient and precise at every single step, not just at the finish line.

4. The "Hard Mode" Test (The Benchmark)

The authors created a new test called a Benchmark. Think of it as a "Final Exam" for shopping AI.

The Setup: It uses 1.2 million real products and creates fake but realistic conversations where the user's preferences are hidden deep in the past (like a "needle in a haystack").
The Result: Even the smartest AI models (like GPT-5) failed this test often, getting less than 70% right. They forgot preferences or messed up the budget.
The Winner: The authors' lightweight model (which is smaller and cheaper to run) used their new training method and beat the big models. It remembered your preferences better and successfully bought the right bundle more often.

Why Does This Matter?

This isn't just about buying shoes. It's about building AI that feels like a real human partner.

It remembers you: It knows your history without you repeating yourself.
It listens to you: It lets you correct it before it makes a mistake.
It handles complexity: It can juggle budgets, coupons, and multiple items at once.

In short, Shopping Companion is the first AI that doesn't just "search" for you; it "remembers" you, "plans" with you, and "shops" for you, all while learning from its mistakes in real-time.

1. Problem Statement

The paper addresses the challenge of deploying Large Language Model (LLM) agents for complex, long-horizon e-commerce tasks (e.g., recommendations, budgeting, bundle deals). While LLMs show promise, two critical gaps hinder their real-world application:

Lack of Benchmarks: Existing benchmarks (e.g., WebShop, ShoppingBench) typically evaluate single-session interactions or isolated memory retrieval. None simultaneously evaluate long-term memory (LTM) across multiple sessions, real-world downstream tasks, and interactive user intervention within a unified framework.
Fragmented Optimization: Current systems treat "preference identification" (memory retrieval) and "shopping assistance" (task execution) as separate modules. This decoupling prevents end-to-end optimization, leading to suboptimal performance where the agent fails to align retrieved memories with final product recommendations.

The core difficulty lies in capturing implicit user preferences (e.g., brand aversions, size history, aesthetic tastes) expressed sparsely over long conversations and applying them to satisfy complex constraints (budget, multi-product bundles) in a dynamic product database.

2. Methodology

A. Novel Benchmark Construction

The authors introduce a new benchmark designed to evaluate preference-grounded shopping success:

Scale: Built on a simulated environment containing 1.2 million real-world products.
Data Synthesis: A five-step pipeline generates 1,000 complex tasks (500 single-product, 500 add-on deals). It involves sampling products, generating user instructions, creating multi-turn dialogues with implicit preferences, and interleaving these with unrelated conversations (needle-in-a-haystack) to test memory retrieval.
Evaluation: Uses an LLM-as-Judge paradigm (GPT-5) with expert-level prompts to verify if recommendations satisfy both explicit user instructions and implicit long-term preferences.

B. The Shopping Companion Framework

The proposed SHOPPING COMPANION is a unified, two-stage agent framework:

Stage 1: Preference Identification: The agent uses memory tools (mem_search, mem_view, mem_summarize_by_date) to retrieve relevant conversation history and extract implicit preferences. Crucially, it presents these extracted preferences to the user for confirmation/intervention before proceeding.
Stage 2: Shopping Assistance: Using the confirmed preferences, the agent iteratively searches the product database (product_search, product_view) to find items that satisfy constraints (budget, bundle compatibility).

C. Training Strategy: Dual-Reward RL with Tool-Wise Supervision

To train the agent in this sparse-feedback, multi-turn environment, the authors developed a specialized Reinforcement Learning (RL) strategy:

Dual-Reward System: The reward function is split into two branches corresponding to the two stages:
- $R_1$ (Stage 1): Rewards accurate grounding of preferences from memory (query relevance, attribute match).
- $R_2$ (Stage 2): Rewards successful task completion (product validity, constraint satisfaction, intent match).
Tool-Wise Rewards ( $R_{tool}$ ): To address the sparsity of terminal rewards, the system assigns intermediate rewards to individual tool calls. It evaluates the quality of each retrieval (e.g., did mem_search return the correct session?) to improve credit assignment.
Format Reward: A lightweight penalty/reward for adhering to structured output protocols (JSON tool calls, Markdown reports).
User Intervention Simulation: The training incorporates a user simulator that provides feedback (hints or corrections) on extracted preferences, allowing the model to learn from coarse to fine-grained corrections.

3. Key Contributions

First Unified Benchmark: Introduced a benchmark that simultaneously evaluates long-term memory, real-world shopping tasks, and multi-turn user intervention, filling a gap in existing literature.
Unified Framework: Proposed SHOPPING COMPANION, which jointly optimizes memory retrieval and task execution, demonstrating that end-to-end training outperforms modular approaches.
Advanced RL Training: Developed a Dual-Reward RL strategy with Tool-Wise supervision, effectively handling sparse and discontinuous feedback in multi-turn agent interactions.
User-Centric Design: Explicitly integrated a mechanism for user intervention, showing that even lightweight models can achieve high performance when allowed to correct preference extraction errors.

4. Experimental Results

The experiments were conducted on a dataset of 1,000 instructions (split into training and test sets) using a lightweight Qwen3-4B model as the backbone.

Baseline Performance: State-of-the-art closed-source models (e.g., GPT-5) achieved a success rate of <70% on the benchmark, highlighting the difficulty of the task. Open-source models performed significantly worse (e.g., Qwen3-4B at ~25% success).
Shopping Companion Performance:
- The proposed method significantly outperformed baselines.
- Single Product Tasks: Achieved 90.0% Accuracy (preference extraction) and 84.0% Success Rate.
- Add-on Deals (Bundle) Tasks: Achieved 55.0% Accuracy and 43.0% Success Rate.
- Comparison: The trained 4B model outperformed larger open-source models (30B/80B) and approached the performance of closed-source giants like GPT-5.
Ablation Studies:
- Two-Stage vs. One-Stage: Separating the tasks into two stages improved success rates by ~23% on bundle deals compared to a single-stage approach.
- Tool-Wise Rewards: Adding tool-wise rewards reduced response verbosity and improved tool utilization quality, leading to more efficient trajectories.
- User Intervention: Simulated user feedback (Low/High hints) progressively improved performance, narrowing the gap to an "Oracle" (perfect memory) baseline.

5. Significance and Implications

Practical Viability: The paper demonstrates that lightweight LLMs, when trained with a unified framework and specialized RL strategies, can outperform larger, untrained models in complex, memory-intensive tasks.
End-to-End Optimization: It validates that treating memory retrieval and task execution as a joint optimization problem is superior to the traditional modular pipeline.
Human-in-the-Loop: The framework's ability to incorporate user intervention suggests a viable path for deploying reliable shopping assistants that can correct their own hallucinations or misinterpretations of user history.
Future Directions: While the method excels at single-product and simple bundles, the paper notes that complex, budget-constrained multi-item optimization remains a significant challenge, pointing to the need for further research in combinatorial reasoning within conversational agents.

In conclusion, Shopping Companion provides a robust blueprint for building next-generation e-commerce agents that are not only capable of searching products but also of remembering user history, adapting to feedback, and executing complex shopping goals with high reliability.