🤖 machine learning

AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing

The paper proposes AIGP, an LLM-based framework that integrates supervised fine-tuning with a Long-Term Value Estimator trained via offline reinforcement learning and Direct Preference Optimization to achieve interpretable, long-term value-aligned dynamic pricing, demonstrating significant improvements in GMV, ROI, and milestone achievement rates in large-scale e-commerce A/B tests.

Original authors: Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao, Keping Yang

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao, Keping Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a massive, bustling digital marketplace, like a giant flea market that never sleeps. Every day, millions of items are listed, and you need to decide the perfect price for each one. If you price too high, no one buys; too low, you lose money. This is the challenge of dynamic pricing.

For a long time, online stores have used two main ways to set prices:

The Rulebook: Humans write strict rules (e.g., "If sales drop, lower price by 10%").
The Math Wizard: Computers use complex math to guess how sales will react to price changes.

The Problem: Both methods are a bit "blind." They can't read customer reviews or product descriptions to understand why people might want an item. They also tend to be short-sighted, chasing a quick sale today even if it hurts the store's reputation or profits next month.

Enter AIGP (Artificial Intelligence Generated Pricing), a new framework created by researchers at Alibaba. Think of AIGP as hiring a super-smart, experienced store manager who has read every book on economics, knows the history of every product, and can talk to you about why they made a decision.

Here is how AIGP works, broken down into simple steps:

1. The "Brain" (The Large Language Model)

Instead of just spitting out a number, AIGP uses a powerful AI (a Large Language Model) that acts like a detective.

The Clues: It looks at hard numbers (how many clicks, how many sales) and soft clues (what customers wrote in reviews, the product description, and even the season, like "Back to School" or "Holiday Season").
The Thought Process: Before giving a price, the AI writes out a "thought process" (like a detective's notebook). It says, "Okay, this desk is great quality, and it's September, so parents are buying school supplies. I should keep the price a bit higher because people are willing to pay more right now."
The Result: It gives a price and a clear explanation, so human managers can trust it.

2. The "Teacher" and the "Student" (Training)

The researchers didn't just let the AI guess. They used a Teacher-Student approach:

The Teacher: A massive, super-smart AI (235 billion "brain cells") that generates perfect pricing strategies and explanations.
The Student: A smaller, faster AI (30 billion "brain cells") that is cheaper to run.
The Lesson: The Teacher teaches the Student how to think and decide. The Student learns to mimic the Teacher's high-quality reasoning but stays small enough to run quickly on millions of products every day.

3. The "Crystal Ball" (Long-Term Value Estimator)

This is the most critical part. Often, AI makes decisions that look good today but are bad next week.

Imagine a seller who slashes prices to zero just to get a sale today. They make a sale, but they ruin their profit for the whole month.
AIGP uses a special tool called the Long-Term Value Estimator (LTVE). Think of this as a Crystal Ball trained on 6 months of real sales data.
Before the AI picks a price, the Crystal Ball simulates: "If we pick this price, what will our total profit be in 14 days?"
If the Crystal Ball says, "This price will hurt us later," the AI learns to avoid it. It teaches the AI to care about the long game (total profit over two weeks) rather than just a quick win.

4. The "Coach" (Preference Optimization)

Once the AI has learned the basics, the researchers act as a Coach.

They show the AI two different pricing choices.
The "Crystal Ball" scores them.
The Coach tells the AI: "You picked the wrong one. The other one would have made us more money in the long run. Remember this for next time."
This process, called Direct Preference Optimization (DPO), fine-tunes the AI to always pick the strategy that wins in the long run.

The Results: Did it work?

The researchers tested AIGP on Tao Factory, a real part of Alibaba's marketplace. They ran a massive experiment for over two months with 200,000 products.

Compared to the old systems, AIGP achieved:

+13.21% more total sales value (GMV): The store sold way more stuff.
+7.59% better return on investment: The store made more profit for every dollar spent.
+8.20% more "Milestones" reached: More products hit their long-term sales goals.

Why is this a big deal?
Unlike previous AI that was a "black box" (you put data in, a number comes out, and you have no idea why), AIGP is transparent. It tells you why it set the price. It also handles "cold start" problems (new products with no history) by using its knowledge of similar items, much like a human expert would.

In short, AIGP is like upgrading from a calculator to a wise, experienced business partner who can read the room, plan for the future, and explain its logic to you.

Technical Summary: AIGP – An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing

1. Problem Statement

Traditional dynamic pricing in large-scale e-commerce faces three critical limitations:

Limited Interpretability: Rule-based systems and black-box data-driven models (e.g., standard Reinforcement Learning) lack transparent reasoning, making it difficult to audit decisions or build merchant trust.
Underutilization of Unstructured Data: Existing methods struggle to leverage rich textual context such as product descriptions, user reviews, and seasonal trends, relying instead on structured historical metrics.
Misalignment with Long-Term Objectives: Aggressive discounting often maximizes short-term sales at the expense of long-term business health. Optimizing for cumulative Gross Merchandise Value (GMV), Return on Investment (ROI), and milestone achievement over extended horizons (e.g., 14 days) remains challenging due to reward sparsity and the difficulty of modeling delayed market effects.

While Reinforcement Learning (RL) offers a path to long-term optimization, it suffers from distributional shifts, reward sparsity, and a lack of interpretability. Conversely, Large Language Models (LLMs) offer reasoning capabilities and unstructured data processing but lack domain-specific operational knowledge and long-horizon outcome feedback, often leading to suboptimal, short-sighted decisions.

2. Methodology: The AIGP Framework

The authors propose AIGP (Artificial Intelligence Generated Pricing), a framework that integrates LLM-based reasoning with offline reinforcement learning to achieve transparent, business-aligned dynamic pricing. The framework operates through a four-stage pipeline:

2.1 Domain-Adaptive Supervised Fine-Tuning (SFT)

To deploy LLMs as pricing policies, the authors employ a teacher-student distillation strategy:

Teacher Model: A large 235B parameter LLM (Qwen3-235B) generates high-quality reasoning traces and pricing decisions based on complex prompts containing structured data (sales history, GMV, ROI), unstructured context (reviews, descriptions), and domain knowledge.
Student Model: A compact 30B parameter model (Qwen3-30B) is fine-tuned via Low-Rank Adaptation (LoRA) to mimic the teacher.
Quality Control: An "LLM-as-Judge" module filters teacher responses based on four dimensions: Data Accuracy, Content Completeness, Reasoning Internal Coherence, and Reasoning-Decision Consistency. Only perfect-score responses are used for training.
Output: The SFT model learns to follow instructions, generate Chain-of-Thought (CoT) reasoning, and output compliant JSON pricing actions.

2.2 Long-Term Value Estimator (LTVE)

To address the lack of long-horizon feedback in LLMs, a dedicated Long-Term Value Estimator (LTVE) is trained via Offline Reinforcement Learning on 5 million historical transitions.

Architecture: A critic-only architecture using a decoupled value learning framework with a double Q-critic ensemble and a state-value network ( $V_\psi$ ).
Reward Design: The reward function balances 14-day milestone achievement (cumulative GMV thresholds) and ROI. It utilizes category-normalized relative rewards to isolate pricing strategy quality from intrinsic product popularity.
Training: The LTVE uses multi-step Temporal Difference (TD) targets ( $n=3$ ) to model delayed market effects without requiring online exploration, mitigating distributional shift risks.

2.3 Preference Alignment via Direct Preference Optimization (DPO)

To align the LLM policy with long-term business objectives, the authors introduce a preference alignment pipeline:

Candidate Sampling: For a given state, the model generates multiple candidate pricing actions in a "decision-only" mode (using an [ACT_ONLY] token).
Pair Construction: The LTVE scores these candidates. Pairs are constructed where the "chosen" action has a significantly higher LTVE score and a distinct discount magnitude compared to the "rejected" action.
Optimization: The student model is fine-tuned using Direct Preference Optimization (DPO) to maximize the likelihood of chosen actions over rejected ones. This step specifically targets the action tokens, ensuring the model learns to select actions that maximize long-term value while retaining the ability to generate full reasoning traces during inference.

2.4 Deployment Architecture

The system is deployed on Tao Factory, a vertical marketplace on Taobao. The workflow involves:

Inference: The aligned LLM receives state data and generates a CoT reasoning trace followed by a structured discount action.
Safety: Actions are constrained by business rules to ensure they fall within safe adjustment bounds ( $A_{safe}$ ).
Transparency: The CoT reasoning provides auditable rationales for pricing decisions, facilitating merchant trust.

3. Key Contributions

Novel Framework: The proposal of AIGP, the first framework to integrate LLM-based reasoning with offline RL-based long-term value modeling for e-commerce pricing.
Preference Alignment Pipeline: A method to automate the generation of high-quality preference pairs for DPO using an offline RL-trained LTVE, effectively bridging the gap between LLM reasoning and long-term business optimization.
Scalable Deployment: A demonstration of distilling complex reasoning and decision-making capabilities from a 235B teacher model into a deployable 30B student model, achieving competitive performance with significantly reduced inference costs.

4. Experimental Results

The framework was evaluated on Tao Factory through extensive offline evaluations and large-scale online A/B tests (60+ days, ~200,000 SKUs).

4.1 Offline Evaluation

Long-Term Value Alignment: The full AIGP (SFT+DPO) achieved the highest average Q-score (2.836) compared to baselines like Price-Sales models (2.564) and RL-DT (2.694).
Operational Fidelity: AIGP achieved the highest Expert Action Matching Accuracy (82.51%) and lowest Mean Absolute Error (0.062).
Reasoning Quality: The distilled 30B model, after SFT and DPO, achieved a reasoning quality score (19.52) approaching the 235B teacher model (19.80), significantly outperforming the base 30B model (18.13).

4.2 Online A/B Test Performance (14-Day Horizon)

Compared to the production baseline (Price-Sales Model), AIGP demonstrated significant improvements:

Milestone Achievement Rate (MAR): +8.20%
Cumulative GMV: +13.21%
Integrated ROI: +7.59%

4.3 Stability and Robustness

Pricing Stability: AIGP exhibited fewer extreme discount jumps and a higher proportion of small-magnitude adjustments compared to baselines, crucial for new product launches.
Cold-Start & OOD: In scenarios with limited history (cold-start) or boundary actions, AIGP maintained high Counterfactual Discrimination Accuracy (94.6% and 94.1% respectively), outperforming both standard LTVE and Conservative Q-Learning (CQL) by leveraging unstructured context and analogical reasoning.

5. Significance and Claims

The paper claims that AIGP successfully addresses the "black box" nature of RL and the "short-sightedness" of generic LLMs in e-commerce pricing. By combining interpretable CoT reasoning with long-term value optimization, the framework enables:

Transparent Decision Making: Merchants can audit the "why" behind pricing changes, fostering trust.
Effective Use of Unstructured Data: The system leverages reviews and descriptions to adjust pricing strategies, particularly for cold-start products where historical data is sparse.
Sustainable Growth: The alignment with long-term metrics (GMV, ROI, milestones) ensures that pricing strategies drive sustainable business health rather than just short-term volume.

The authors conclude that this approach allows compact, deployable models to match the decision quality of much larger LLMs while ensuring policy transparency and alignment with complex, multi-objective business goals. Future work is suggested to explore cross-domain extensibility and unified frameworks for symbolic reasoning and strategic decision-making.