💻 computer science

Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework

This paper proposes a multi-objective framework for automatically optimizing prompts in LLM-based user simulators for conversational recommender systems, effectively addressing challenges related to evaluation costs, data scarcity, and systematic biases while improving alignment with human interaction patterns.

Original authors: Nipun B Nair, Tongtong Wu, Weiqing Wang

Published 2026-07-02✓ Author reviewed ⓘ

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nipun B Nair, Tongtong Wu, Weiqing Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a very smart robot that acts as a movie recommender. Before you let this robot talk to real people, you need to test it to make sure it's good at suggesting movies and, more importantly, good at handling it when people say, "No, I don't like that one."

The problem is that testing with real humans is expensive, slow, and raises privacy issues (you don't want to record everyone's private movie tastes). So, researchers usually build a "fake human" (a simulator) to test the robot.

However, the current "fake humans" built by AI are terrible at acting like real people. They are like overly polite guests who say "Yes!" to every single movie suggestion, even if they hate it. They also tend to only suggest famous blockbusters and accidentally "cheat" by repeating movies they were told about in their backstory.

This paper introduces a new automatic training method to fix these "fake humans" so they act more like real people. Here is how they did it, using simple analogies:

1. The Problem: The "Yes-Man" Robot

Current AI simulators suffer from three main glitches:

The "Yes-Man" Bias: They accept almost every recommendation, even bad ones. Real people reject about half the suggestions; these bots say "Yes" to everything.
The "Leak" Problem: If you tell the bot, "I love The Godfather," and then ask it to recommend a movie, it might just say, "How about The Godfather?" instead of thinking of a new one. It's cheating by repeating the answer it was given.
The "Boring" Bias: They only talk about popular, mainstream movies, ignoring niche or unique films that real humans actually enjoy.

2. The Solution: The "Auto-Coach"

Instead of a human spending hours writing strict rules (prompts) to tell the AI how to behave, the authors built an automatic coach (using a tool called TextGrad).

Think of this coach like a video game trainer.

The Old Way: A human coach writes a long, rigid rulebook: "If the user says no, you must say no. If they ask for a thriller, pick a thriller." This is hard to get right and breaks easily.
The New Way: The coach watches the AI play the game. If the AI says "Yes" to a movie it should have hated, the coach gives it a "textual nudge" (a gradient) saying, "That was too nice, try being more critical." If the AI repeats a movie title, the coach says, "Don't say the name, describe the genre instead."
The AI learns from these nudges automatically, refining its "personality" over a few rounds until it acts like a real human.

3. The "Summary" Trick (Stopping the Cheating)

To stop the AI from cheating by repeating movie titles from its backstory, the researchers introduced a Summary Step.

Before: They gave the AI a list of 50 movies it liked. The AI would just copy-paste one of those titles.
Now: They ask the AI to summarize that list into a sentence like, "I love crime dramas with moral ambiguity."
Result: The AI now has to think about what it likes rather than just copying a title. This stops the "leakage" and forces it to be more creative.

4. The New Test: The "Rejection Score"

The authors realized that just checking if the AI says "Yes" or "No" isn't enough. They created a new test called NegFeedback.

Imagine a movie recommender suggests a horror movie to someone who hates horror.
Bad Simulator: "Oh, I don't like that." (Too simple).
Good Simulator: "No thanks, I'm not into scary movies because I prefer mysteries with clever puzzles." (Specific, in character, and helpful).
The authors built a scoring system (and even used another AI to grade it) to see if the simulator could reject movies politely and explain why, just like a real person would.

5. The Results: From "Robo-Polite" to "Real Human"

When they tested their new method against standard AI models (like GPT-3.5 and GPT-4):

Diversity: The new simulator suggested a much wider variety of movies (from different countries and eras), whereas the old ones stuck to the same popular hits.
Rejection: The new simulator learned to say "No" realistically. It stopped being a "Yes-Man" and started giving reasons for its dislikes.
Accuracy: Its behavior matched real human data much better than the standard models, which tended to be either too positive or too repetitive.

Summary

The paper presents a system that automatically teaches AI simulators to act like real humans by fixing their tendency to be too nice, too repetitive, and too boring. It does this by using an automatic coach to tweak the instructions and a "summary" trick to prevent cheating. The result is a simulator that can test movie recommenders more realistically without needing to interview thousands of real people.

Technical Summary: Prompt Optimization for User Simulation in Conversational Recommender Systems

1. Problem Statement

Conversational Recommender Systems (CRSs) are critical for enabling users to articulate preferences and clarify intentions in real-time. However, their development faces two primary bottlenecks: the high cost and time consumption of real human evaluation studies, and the difficulty of obtaining training data due to privacy concerns. While Large Language Model (LLM)-based user simulators offer a scalable, privacy-preserving alternative, existing approaches suffer from three critical limitations that hinder their practical deployment:

Systematic Positive Bias: Simulators exhibit unrealistically high acceptance rates, failing to capture realistic rejection behaviors. They tend to accept recommendations regardless of stated preferences, unlike real users who reject a significant portion of suggestions.
Data Leakage: Simulators often "echo" items from a user's profile history verbatim in their responses. This occurs when target items appear in the history, leading to inflated accuracy metrics and artificially suppressed diversity, misrepresenting the simulator's true capability.
Constrained Behavioral Diversity: Current simulators display popularity bias (favoring mainstream items) and temporal clustering, resulting in homogenized interactions that fail to reflect the diverse preferences of real user populations.

Furthermore, existing prompt-based simulators rely on brittle manual prompt engineering, which requires extensive domain expertise and lacks systematic methods for optimization.

2. Methodology

The authors propose a multi-objective framework for the automatic optimization of prompts for LLM-based user simulators. The framework leverages TextGrad, an automated prompt optimization technique that uses textual gradients to iteratively refine prompts without modifying model weights. The system is implemented using a local deployment of Llama-3.3-70B via Ollama to ensure privacy-preserving execution.

The framework addresses the three core challenges through specific design components:

A. Bias Correction via Entropy-Aware Scoring

To mitigate positive and popularity bias, the framework introduces entropy-based metrics that guide the prompt optimization process. Rather than simply maximizing entropy (which could lead to random behavior), the system optimizes against empirically derived human baseline distributions. Key metrics include:

Item Entropy: Measures diversity in movie selections across genre, era, region, and cultural origin, penalizing duplicates and overuse of top-rated lists (e.g., IMDb Top 250).
BinPref: Calculates the Pearson correlation between simulated acceptance rates and actual human ratings to ensure preference alignment.
Aspect & Sentiment Entropy: Measures the diversity of justifications (e.g., plot, acting) and tone variation to expose and correct positive bias.
Semantic Richness: Assesses vocabulary diversity in recommendation requests.

B. Profile Summarization for Data Leakage Prevention

To prevent data leakage, the framework replaces raw user history with profile summaries. Instead of passing specific movie titles to the LLM, the system samples up to $k=5$ items from a user's history and prompts the LLM to generate a concise, natural language description of preferences (e.g., "prefers character-driven crime dramas") without naming specific titles. This abstracts sensitive data while retaining essential behavioral traits.

C. NegFeedback: A Novel Evaluation Metric

The authors introduce NegFeedback, a proof-of-concept metric designed to evaluate the quality of negative feedback (rejections). Unlike prior work that focuses on acceptance coherence, NegFeedback assesses:

Rejection Clarity: The explicitness of the refusal.
Justification Specificity: Whether the rejection is grounded in the user's persona preferences.
Reformulation Quality: The ability to request alternative recommendations semantically without naming specific items.
Tone Consistency: Alignment with the user's persona.
This metric is evaluated using an LLM-based scorer (Llama-3.3-70B) trained on a rubric, which is validated against human evaluators.

D. Optimization Loop

The framework employs an iterative optimization loop:

Initialize a task-specific prompt.
Generate responses from a batch of users.
Compute task-specific metrics (e.g., entropy, correlation).
Query the TextGrad engine to generate textual feedback on how to improve the metrics.
Update the prompt (specifically the guideline and persona sections) based on this feedback.
This process runs for a fixed number of iterations (e.g., $T=3$ ) to refine the simulator's behavior.

3. Key Contributions

Automated Prompt Optimization Framework: The first framework to automatically optimize prompts for LLM-based user simulators in CRSs, simultaneously addressing positive bias, data leakage, and behavioral diversity.
Novel Metrics and Solutions:
- Design of entropy-aware and textual-gradient-based scoring functions to correct positive and popularity biases.
- Implementation of profile summarization to reduce data leakage while preserving essential user information.
- Proposal of NegFeedback, a metric specifically designed to assess the correctness and rationale of negative user feedback.
Improved Behavioral Alignment: Demonstration of superior alignment with human interaction patterns compared to GPT-3.5 and GPT-4 baselines, validated using local Llama-3.3 execution.
Scalable Evaluation: Validation of an LLM-based evaluator for NegFeedback, showing alignment with human judgments, which enables scalable assessment of rejection behavior without exhaustive human annotation.

4. Experimental Results

The framework was evaluated on the Amazon Reviews 2023 (Movies & TV) dataset across five tasks: ItemsTalk (Diversity), BinPref (Preference Alignment), OpenPref (Review Richness), RecRequest (Semantic Richness), and NegFeedback (Rejection Quality).

ItemsTalk (Diversity): The optimized method achieved an item entropy of 9.269, closely matching the human baseline of 10.067 and significantly outperforming GPT-3.5 and GPT-4 baselines (which ranged from 6.49 to 6.64).
BinPref (Preference Alignment): The method achieved a Pearson correlation of 0.726 with human preferences, a substantial improvement over GPT-3.5 (0.079–0.265) and GPT-4 (0.265).
OpenPref (Review Richness): Post-tuning aspect entropy reached 7.523 (vs. human 6.761), while sentiment entropy was 1.274 (vs. human 1.277). In contrast, GPT baselines collapsed to 0.000 sentiment entropy, indicating a severe lack of negative sentiment expression.
RecRequest (Semantic Richness): The method achieved a Type-Token Ratio (TTR) of 0.235, closely approximating the human baseline of 0.188, whereas GPT baselines exhibited inflated diversity (0.42+), suggesting unnatural lexical variety.
NegFeedback (Rejection Quality): The proposed method achieved a human-LLM correlation of 0.484, outperforming base Llama (0.258) and all GPT-3.5 variants (which showed negative or near-zero correlations).

5. Significance and Claims

The paper claims that its framework provides a principled, automated solution to the "brittle" nature of manual prompt engineering in user simulation. By jointly optimizing for behavioral fidelity rather than just linguistic similarity, the approach successfully mitigates systematic biases and data leakage issues that currently plague LLM-based simulators.

The authors emphasize that their work addresses a core challenge in LLM-driven data engineering: the ability to automatically generate, optimize, and validate synthetic behavioral data at scale. The integration of profile summarization and the NegFeedback metric offers a pathway for privacy-compliant, reproducible simulation pipelines suitable for regulated industries (e.g., healthcare, finance) where real user data is inaccessible. The framework is presented as immediately applicable for practitioners seeking to evaluate CRSs without the prohibitive costs of human studies, provided they have access to local LLM infrastructure.

Note: The paper acknowledges the use of AI tools (Claude) for refining language and clarity but maintains that all technical content, analysis, and final responsibility for accuracy rest with the authors.