📈 economics

Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion

This paper establishes a tractable framework analyzing the strategic trade-offs between Supervised Fine-Tuning and In-Context Learning for LLM personalization under resource congestion, revealing that offering both methods is always profitable for platforms while user equilibrium choices exhibit non-monotonic sensitivity to pretraining quality and task difficulty.

Original authors: Fengzhuo Zhang, Zhuoran Yang, Dirk Bergemann

Published 2026-07-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Fengzhuo Zhang, Zhuoran Yang, Dirk Bergemann

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a tricky puzzle, but instead of doing it alone, you have a super-smart robot friend who has read almost every book in the library. This robot is a Large Language Model (LLM). It's great at general knowledge, but sometimes it gets stuck on very specific tasks, like analyzing a rare medical condition or decoding a specific financial trend. To help the robot, you can give it a "cheat sheet" of examples right in the conversation (this is called In-Context Learning or ICL). It's quick and easy, like whispering a hint to a friend. Or, you can take the time to retrain the robot's brain specifically for your task using a huge dataset (this is called Supervised Fine-Tuning or SFT). This is powerful and makes the robot a true expert, but it's heavy, slow, and costs a lot of energy.

Now, imagine millions of people are trying to use these robots at the same time on a single, shared computer network. If everyone tries to retrain their robot at once, the network gets clogged, like a highway during rush hour. Everyone has to wait longer, and the cost of using the service goes up. This paper asks a fascinating question: When you are part of this crowded digital highway, should you whisper a hint (ICL) or retrain the brain (SFT)? Does the fact that everyone else is making a choice change what is best for you? The authors treat this like a giant game of strategy, where your choice affects the traffic for everyone else, and they use math to figure out the perfect balance between getting a great answer and not getting stuck in a digital traffic jam.

The Great Robot Debate: Whispering Hints vs. Rewiring Brains

So, you've got a million users, a limited amount of computer power, and two ways to make a smart AI work better for you. The authors of this paper built a mathematical model to see how this plays out. They didn't just guess; they created a "continuum" of users (think of it as a smooth, endless stream of people rather than just a few individuals) and watched how they behaved when they had to choose between the lightweight In-Context Learning (ICL) and the heavy-duty Supervised Fine-Tuning (SFT).

Here is the twist: The "best" choice isn't the same for everyone. It depends on two main things: how much the robot already knows about your specific topic (pretraining coverage) and how clear the signal is in your data (signal-to-noise ratio).

The "Goldilocks" Zone of Personalization
The paper finds that if the robot's pre-training already covers your topic well, and your data is clear, SFT (the heavy retraining) is the winner. It's like hiring a specialist to fix your car; if you have the right tools and a clear problem, they can fix it perfectly. However, if the robot knows very little about your topic, or your data is messy and noisy, ICL (the whispering hints) actually wins. Why? Because trying to retrain the robot on a topic it knows nothing about, with messy data, is like trying to teach a fish to climb a tree. You might just confuse the robot and make it worse. In these cases, it's safer to just give it a few examples in the chat and let it do its best with what it already knows.

The Traffic Jam Surprise
The most surprising part of the story is how "congestion" (the traffic jam) changes the rules. The authors discovered that the relationship between how good the robot is and how much traffic it causes isn't a straight line.

Better Pre-training usually helps: If the robot starts with a better brain (higher precision), people use less computing power, and the traffic jam gets smaller.
But more coverage can cause jams: If the robot learns about more topics (broader coverage), it might actually make the traffic worse in some situations. It's like if a highway suddenly adds more lanes; people might decide to drive more because the road looks easier, eventually causing a new kind of gridlock.
Harder tasks can clear the road: If a task is really difficult and noisy, people might just give up on personalizing the robot entirely to avoid the cost, which actually reduces the traffic!

The Platform's Dilemma
Finally, the paper looks at the company running the AI service (the platform). They want to make money, so they set prices. The authors prove something counter-intuitive: Offering both options (SFT and ICL) never hurts the platform's profit. Even though adding SFT might increase the total load on the servers, it gives users more choices. Some users will switch to the heavy SFT option when it's cheap, increasing the demand, while others stick to ICL. The platform can always find a price point where they make the most money, regardless of whether they offer one or both methods. In fact, the data shows that the number of platforms offering both SFT and ICL jumped from 9.5% in 2021 to 71.4% in 2025, suggesting that the market is already moving exactly where the math predicts it should go.

The Proof in the Pudding
To make sure their math wasn't just pretty theory, the authors tested it. They trained a model called GPT-2 on simple math problems (linear regression).

They found that when they gave the model more examples (ICL), the error stopped getting better after a certain point, hitting a "ceiling" caused by the parts of the problem the model hadn't seen before.
They also saw the "flip" they predicted: with very few examples, ICL was better; with hundreds of examples, SFT took over.
They even checked real-world data from 21 major AI companies and confirmed that the trend toward offering both methods is real and growing.

In short, this paper tells us that personalizing AI isn't just about picking the "best" tool; it's about a complex dance between what the AI already knows, how messy your data is, and how crowded the digital highway is. The "right" choice depends on the whole system, not just your own needs. And for the companies running the show? Offering a menu of options is always the smartest move.

Technical Summary: Supervised Fine-Tuning vs. In-Context Learning under Congestion

Problem Statement

Large Language Models (LLMs) have revolutionized AI services, yet a critical economic tension exists between personalization and resource scarcity. While personalization (via Supervised Fine-Tuning, SFT, or In-Context Learning, ICL) improves model performance on domain-specific tasks, it consumes shared computational resources. When thousands of users simultaneously personalize models on shared infrastructure, congestion emerges: each user's choice of method and sample size affects not only their own costs but also the waiting times and latency for others.

The paper addresses three primary questions:

Algorithm Choice: In the absence of congestion, how should users choose between SFT and ICL based on statistical primitives?
Equilibrium Behavior: With shared, constrained resources, how do users trade off personalization quality against congestion costs in equilibrium?
Platform Design: When resources are scarce, how should platforms design their service menus (offering SFT, ICL, or both) and pricing strategies?

Existing literature often relies on scaling laws or stylized behavioral assumptions, missing the crucial differences between personalization algorithms and the strategic interactions at the scale of millions of users. This paper aims to bridge the gap between statistical learning theory and congestion economics.

Methodology

The authors develop a tractable framework combining statistical modeling of LLMs with a continuum-user game-theoretic model.

1. Statistical Models of Personalization

The paper models LLM personalization as a linear regression problem.

Pretraining: Abstracted as learning a prior distribution over task parameters $\theta$ from a dataset of size $N$ . The pretrained model corresponds to a learned prior mean $\theta_{pre}$ and covariance $\Sigma_{pre}$ .
In-Context Learning (ICL): Modeled as approximate Bayesian inference. The model updates its belief about the target task $\theta^*$ by conditioning the learned prior on $N$ personalization samples provided in the prompt, without updating parameters.
Supervised Fine-Tuning (SFT): Modeled as a regularized estimator that updates model parameters using task-specific data, starting from the pretrained initialization. This incurs higher computational costs (FLOPs and memory) but allows adaptation beyond the pretrained subspace.

The authors derive closed-form expressions for the mean-squared error (MSE) of both methods, decomposing errors into components inherited from pretraining, data noise, and algorithm-specific uncertainty.

2. Continuum User Model (LLM-Serving Congestion Game)

The interaction is modeled as a LLM-Serving Congestion Game (LSCG) with a continuum of users.

User Types: Users are characterized by a type $t$ summarizing the statistical properties of their target task and the pretrained model (e.g., dimension $d$ , pretraining coverage $r$ , signal-to-noise ratios).
Actions: Each user chooses a personalization algorithm $a \in \{ICL, SFT\}$ and a number of samples $\tilde{N}$ .
Cost Function: A user's total cost comprises the learning error (statistical performance) and the resource cost. The resource cost includes a monetary price per unit of resource and a congestion-induced time cost $h(R)$ , where $R$ is the aggregate resource demand.
Equilibrium: The paper defines a Nash equilibrium in this continuum setting, where almost every user minimizes their cost given the aggregate congestion level, and the congestion level is consistent with the aggregate behavior of all users.

Key Contributions and Results

1. Statistical Trade-offs: SFT vs. ICL

The analysis yields a precise threshold for when SFT dominates ICL, governed by pretraining coverage ( $R = r/(d-r)$ ) and the signal-to-noise ratio (SNR) of the personalization data in the uncovered subspace ( $\kappa$ ).

Finding: ICL is optimal when pretraining coverage is low and data is noisy/limited. In this regime, ICL's conservatism (updating only within the pretrained subspace) avoids the high variance of SFT attempting to learn in poorly supported directions.
Finding: SFT dominates when pretraining coverage is high and data is informative. SFT can reduce error in directions not covered by pretraining, whereas ICL incurs a fixed irreducible bias in those directions.
Validation: Experiments with GPT-2 on linear regression tasks confirm that ICL error plateaus at a bias determined by uncovered dimensions, while SFT error continues to decrease with sample size.

2. Equilibrium Congestion Dynamics

The paper proves the existence and uniqueness of the equilibrium congestion level $R^*$ . Crucially, the relationship between system parameters and congestion is non-monotonic:

Pretraining Precision: Higher precision ( $\pi$ ) always reduces congestion, as users rely less on expensive online personalization.
Pretraining Coverage: Increasing coverage ( $r$ ) can increase congestion in intermediate regimes. As coverage expands, users may switch from ICL to SFT (extensive margin) or increase sample sizes (intensive margin) to exploit the newly covered dimensions, temporarily raising aggregate demand.
Task Difficulty: Harder tasks (high noise) can reduce congestion by deterring users from personalizing altogether.
SFT Cost: Increasing the resource cost of SFT can initially increase congestion (as users intensify usage before switching) before eventually reducing it as users substitute toward ICL.

3. Platform Strategy and Profitability

The paper analyzes the platform's Stackelberg strategy (setting price $p$ to maximize profit).

Pricing: The equilibrium congestion level is decreasing in the unit resource price. The profit-maximizing price is finite under regularity conditions.
Menu Design: The authors prove that offering both SFT and ICL never reduces the platform's maximal profit compared to offering only ICL. While adding SFT increases the potential for higher aggregate resource consumption (and thus congestion), it allows the platform to capture surplus from users who prefer the high-performance, high-cost option.
Empirical Evidence: A review of 21 major AI platforms shows the share offering both SFT and ICL increased from 9.5% in 2021 to 71.4% in 2025, consistent with the theoretical prediction that diversity in personalization options is profitable.

Significance and Claims

The paper claims to contribute to the emerging AI-economics literature by developing a behaviorally grounded model that moves beyond per-token pricing to capture how algorithmic characteristics shape strategic interactions.

Theoretical Novelty: It connects classic congestion game theory (peak-load pricing) with the specific statistical properties of LLMs. Unlike traditional infrastructure congestion driven by physical network constraints, this paper highlights a new source of congestion driven by the algorithmic properties of personalization methods (e.g., how SFT and ICL differ in resource intensity and error reduction).
Practical Relevance: The framework explains the rapid adoption of SFT APIs by major platforms despite the added complexity and load. It provides a theoretical justification for why platforms should offer a menu of personalization options rather than a single standardized method.
Policy Implications: The non-monotonic nature of congestion suggests that simply improving pretraining models or increasing resources may not linearly reduce congestion; platform design must account for how users shift between algorithms and sample sizes in response to these changes.

The authors validate their theoretical predictions using GPT-2 experiments and industry data, demonstrating that the linear approximation of LLM behavior effectively captures the core trade-offs between SFT and ICL in a congested environment.