💻 computer science

SimPersona: Learning Discrete Buyer Personas from Raw Clickstreams for Grounded E-Commerce Agents

SimPersona is a novel framework that learns discrete, interpretable buyer personas from raw clickstream data using a VQ-VAE to ground LLM-based e-commerce agents in realistic population distributions, achieving superior conversion-rate alignment and goal-oriented performance compared to existing methods without requiring retraining or manual prompt engineering.

Original authors: Zahra Zanjani Foumani, Alberto Castelo, Shuang Xie, Ted Chaiwachirasak, Han Li, Lingyun Wang

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zahra Zanjani Foumani, Alberto Castelo, Shuang Xie, Ted Chaiwachirasak, Han Li, Lingyun Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to shop online. If you just tell the robot, "Go buy some shoes," it will act like the "average" person. It might browse a little, maybe add one thing to the cart, and then leave. But in the real world, shoppers are wildly different. Some are "window shoppers" who just look and never buy. Some are "deal hunters" who search for hours. Others are "impulse buyers" who grab the first thing they see and check out immediately.

The problem with current AI shopping agents is that they only know how to be that "average" person. They can't switch personalities to match the specific type of human they are supposed to represent.

Enter SimPersona: The "Personality Chip" for Shopping Robots

The researchers at Shopify created a new system called SimPersona. Think of it as a way to give a shopping robot a specific "personality chip" that tells it exactly how to behave, based on real data from millions of actual shoppers.

Here is how it works, broken down into simple steps:

1. The Detective Work (Learning from the Crowd)

First, the system looks at raw clickstreams. Imagine a giant pile of receipts and logs showing what millions of people clicked, how long they stared at a page, what they added to their cart, and what they abandoned.

Instead of asking humans to write descriptions like "The Impulsive Buyer" or "The Careful Researcher," the computer uses a special math tool (called a VQ-VAE) to find patterns in this messy data. It's like a detective sorting through a million fingerprints and realizing, "Hey, these 50,000 people all click the same way, and these 30,000 others do something totally different."

The system groups these millions of shoppers into discrete types (or "personas"). It doesn't just guess; it learns the statistical "fingerprint" of each type.

2. The ID Card (The Persona Token)

Once the system has identified these different shopper types, it gives each one a unique ID card, which the researchers call a "persona token."

Think of this token like a tiny, invisible sticker you can stick onto the robot's brain.

If you stick the "Window Shopper" token on the robot, it will browse slowly and rarely buy.
If you stick the "Super Buyer" token on it, it will move fast, add things to the cart, and checkout quickly.

The best part? These tokens are tiny. They take up almost no space in the robot's memory, but they carry a huge amount of behavioral information.

3. The Training (Teaching the Robot to Act)

Just having the ID card isn't enough; the robot needs to learn how to act like that person. The researchers used a two-step training process:

Step 1: Learning the Vibe. First, they freeze the robot's brain and only teach it what each token means. They show it examples of how real people with that token behave, so the robot learns the "personality" without getting confused by the actual shopping instructions.
Step 2: Learning the Moves. Next, they unlock the robot's brain and teach it how to actually navigate the website using that personality. Now, when the robot sees a "Buy" button, it knows whether to click it immediately (if it has the "Impulse" token) or to keep looking (if it has the "Researcher" token).

4. The Result: A Crowd of Unique Shoppers

When the system is ready, they can simulate a whole store full of shoppers. Instead of having 1,000 robots that all act the same, they can assign different tokens to each robot.

They can create a crowd that looks exactly like the real customer base of a specific store.
If a real store has 80% window shoppers and 20% buyers, the simulation will naturally have that same mix.
The robots don't need to be retrained for every new store; the tokens work anywhere.

Why This Matters (According to the Paper)

The researchers tested this on 8.37 million real buyers across 42 different online stores. Here is what they found:

It's Accurate: The robots mimicked real human buying rates with 78% accuracy. They didn't just guess; they matched the real conversion rates (how many people actually buy) almost perfectly.
It's Smarter than Bigger Models: Even though their system is much smaller than other massive AI models, it performed better at following shopping instructions.
It's Stable: Without these personality tokens, the robots often crashed or got confused. With the tokens, they navigated the websites much more smoothly, just like a human would.

In a Nutshell:
SimPersona is a way to turn messy data from millions of shoppers into a set of "personality chips." By plugging these chips into a shopping robot, you can make it act exactly like a specific type of human buyer, allowing companies to test their websites and marketing strategies with a realistic crowd of digital shoppers, rather than just one boring, average robot.

Technical Summary: SimPersona

Problem Statement

Large Language Model (LLM)-based web agents have demonstrated the ability to navigate live e-commerce storefronts and execute shopping flows. However, current agents typically collapse into a single "average buyer" policy, failing to capture the heterogeneous and distributional nature of real buyer populations. Existing personalization methods rely on hand-crafted, prompt-based personas (e.g., role-playing profiles or purchase summaries) which are brittle, context-inefficient, and unable to faithfully represent population-level behavior. Furthermore, while user modeling literature can learn structured buyer representations from raw clickstreams, these methods are designed for offline tasks (prediction, clustering) and do not "ground" these representations into the action space of a closed-loop agent. Consequently, reconstructing the specific distribution of buyer types that defines a store's traffic remains an open challenge.

Methodology

The authors propose SimPersona, a framework that learns discrete buyer types from historical traffic and exposes them to LLM-based web agents as compact persona tokens. The approach consists of three core components:

1. Data Pipeline

The system processes raw, fragmented e-commerce event logs (page views, cart actions, searches) into two complementary data types:

Buyer Behavioral Representations: Raw logs are enriched with product catalogs and search queries to create aggregated buyer–session vectors. These vectors combine 16 scalar behavioral features (exposure, engagement, funnel rates, intent strength, dollar values) with semantic product embeddings (compressed via PCA). Buyers are also stratified into five funnel levels (Purchasers, Checkout Abandoners, Cart Builders, Window Shoppers, Bouncers).
Agent Traces: Real buyer sessions are replayed on live storefronts. An LLM rewrites the event sequence into a natural-language navigation goal, and a separate agent executes this goal, generating multi-turn sequences of DOM observations, actions, and reasoning traces for Supervised Fine-Tuning (SFT).

2. Persona Discovery via Behavior-Aware VQ-VAE

A Vector-Quantized Variational Autoencoder (VQ-VAE) maps the high-dimensional behavioral vectors to a discrete codebook of $K$ tokens.

Architecture: The encoder compresses inputs into a latent space, which is quantized to the nearest entry in a learned codebook. The decoder reconstructs the input.
Behavior-Aware Objective: Unlike standard clustering (e.g., k-means), the VQ-VAE is trained with a composite loss function including:
- Reconstruction Loss: Group-aware (MSE for scalars, cosine distance for embeddings).
- Contrastive Loss (InfoNCE): A three-stage gating mechanism (funnel level, product similarity, behavioral style) ensures that buyers assigned to the same token share meaningful behavioral similarities, not just geometric proximity.
- Auxiliary Heads: Three classification heads predict coarse labels for engagement depth, exploration breadth, and purchase intensity, enforcing semantic clustering.
Outcome: The frequency of token assignments naturally mirrors the empirical distribution of buyer types within a specific merchant's traffic.

3. Two-Stage Persona Grounding

To integrate these discrete tokens into an LLM (Qwen3-14B-Base) without causing the model to ignore the tokens in favor of surface-level prompt cues, a two-stage fine-tuning process is employed:

Stage 1 (Persona Grounding): The LLM backbone is frozen; only the new persona token embeddings are trained. The model is trained on intent-neutral goals (e.g., "You are interested in skirts") where the only signal distinguishing buyer types is the statistical pattern of actions across traces. This forces the embeddings to absorb the behavioral meaning of the persona.
Stage 2 (Action-Oriented Fine-Tuning): The backbone is unfrozen, and the model is trained on goal-directed traces with explicit intents (e.g., "You are here to buy skirts"). The model learns to fuse the persona token (general behavioral profile) with the specific session intent.

Key Contributions

SimPersona Framework: A method to learn discrete persona tokens from raw clickstreams using a behavior-aware VQ-VAE, enabling scalable persona assignment and faithful reconstruction of buyer population distributions.
Two-Stage Grounding: A training procedure that decouples learning the semantic meaning of a token from learning how to act on it, producing embeddings that generalize across unseen storefronts without adaptation.
Open-Source Pipeline: A data pipeline that converts raw e-commerce event logs into buyer representations and agent-training traces.
Evaluation: Comprehensive testing on 8.37 million unseen buyers across 42 live storefronts.

Results

Evaluated on 42 held-out live storefronts with 8.37M buyers, SimPersona demonstrates:

Conversion Alignment: The agents achieve 78% alignment with real buyer conversion rates (Add-to-Cart and Purchase), significantly outperforming mismatched baselines.
Behavioral Fidelity: Agents conditioned on specific tokens exhibit statistically significant separation in behavioral dimensions (e.g., High-purchase tokens convert at rates >100x higher than Low-purchase tokens under the same intent).
Instruction Following: SimPersona outperforms a baseline model with 8x more parameters (GPT-OSS-120B) on goal-oriented shopping tasks, particularly in complex checkout flows, while maintaining robustness across diverse storefronts.
Distribution Recovery: The learned token distribution accurately recovers the funnel-stage composition of stores (mean Jensen-Shannon divergence of 0.054) and continuous behavioral features ( $R^2 \ge 0.87$ ).
Stability: The two-stage training eliminates catastrophic failure modes seen in single-stage training, reducing the maximum per-shop error rate from 71.3% to 28.5%.

Significance

The paper claims that SimPersona bridges the gap between offline user modeling and closed-loop agent simulation. By representing buyer behavior as discrete tokens learned directly from traffic, the framework solves the "persona problem" not just for individual agents, but for the population as a whole. It allows for the simulation of realistic, heterogeneous buyer populations that preserve merchant-specific traffic distributions, enabling more accurate storefront evaluation and synthetic A/B testing. The work demonstrates that discrete, learned tokens can effectively steer LLM agents to reflect specific buyer segments without the brittleness or scalability issues of prompt-based conditioning.