📈 economics

Nonparametric Identification of Demand without Exogenous Product Characteristics

This paper demonstrates that nonparametric identification of differentiated product demand and price counterfactuals is achievable using market-level data with endogenous product characteristics by employing recentered instruments under a weaker index restriction and a "faithfulness" condition, challenging the conventional view that exogenous characteristic-based instruments are strictly necessary.

Original authors: Kirill Borusyak, Jiafeng Chen, Peter Hull, Lihua Lei

Published 2026-02-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Kirill Borusyak, Jiafeng Chen, Peter Hull, Lihua Lei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out how much people love a specific product, like a new type of smartphone. You want to know: "If we raise the price by $100, how many fewer people will buy it?"

In the world of economics, this is called estimating demand. Usually, to solve this mystery, detectives (economists) look for two things:

Price Shocks: Something that changes the price randomly (like a sudden tax or a supply chain glitch).
Product Features: Things about the phone that are fixed and unrelated to the market, like the color or the screen size, which they assume are chosen randomly by the manufacturer.

The Old Rule:
For a long time, the "conventional wisdom" (the old rulebook) said: You cannot solve this case unless you have both. If the manufacturer strategically chooses the phone's features (e.g., they make the phone blue because they know people in that city love blue and will pay more), the old rulebook said your investigation was doomed. You couldn't tell if people bought it because of the price or because of the blue color.

The New Discovery:
This paper by Borusyak, Chen, Hull, and Lei says: "Not so fast! We can still solve the case, even if the product features are suspicious."

Here is the simple breakdown of their breakthrough, using a few metaphors:

1. The "Recentered Instrument" (The Smart Filter)

Imagine you are trying to hear a whisper (the true effect of price) in a noisy room.

The Noise: The product features (like the blue color) are noisy because the manufacturer might be manipulating them.
The Whisper: The price changes.

The authors suggest a clever trick called "Recentering."
Think of it like this: You know the manufacturer usually picks a "standard" blue for 90% of phones. But sometimes, due to a random supply glitch (your price instrument), they are forced to use a slightly different shade of blue.
Instead of looking at the blue color itself (which is messy), you look at how much the blue color deviates from the average, after you account for the price shock.

You are essentially saying: "Okay, I know the price went up. Now, looking at the features, did they change in a way that has nothing to do with the price?" By mathematically "centering" the data around the average, they cancel out the strategic manipulation. It's like using noise-canceling headphones to isolate the whisper.

2. The "Faithfulness" Condition (The Trustworthy Proxy)

This is the paper's most famous new concept. They call it "Faithfulness."

Imagine you are trying to guess a person's mood (the hidden "demand shock") just by looking at their outfit (the product features).

The Problem: Sometimes people dress up because they are happy, but sometimes they dress up because they are going to a wedding (a strategic choice). It's confusing.
The Faithfulness Solution: The authors say, "We don't need the outfit to be randomly chosen. We just need the outfit to be a strong, reliable clue."

If the outfit always changes in a predictable way when the mood changes (even if the person is trying to hide it), then the outfit is a "faithful proxy."

The Metaphor: Think of a dog. If a dog barks, it usually means it's excited. But sometimes it barks because it sees a squirrel. If you know the dog is faithful to its excitement (it barks every single time it's excited, even if it barks for other reasons too), you can still figure out if it's excited by listening to the bark, as long as you have a way to filter out the squirrel noise.

In this paper, the "faithful proxy" is the product characteristic. It doesn't have to be innocent; it just has to be predictably linked to the hidden demand.

3. The "Magic Transformation"

Here is the coolest part. The authors prove that even if you can't perfectly figure out exactly how much people love the phone (the full demand curve), you can perfectly figure out what happens if you change the price.

The Analogy:
Imagine you are trying to measure the volume of a mystery liquid in a cup, but the cup is made of a stretchy, rubbery material that changes shape.

You can't know the exact volume (the full demand function) because the cup is stretching weirdly.
However, if you pour in more water (change the price), you can still predict exactly how much the water level will rise, even if you don't know the cup's exact shape.

The math shows that the "stretchy shape" (the unknown part of the demand) cancels out when you calculate price changes. So, you get the answer you actually care about (the price effect) without needing to solve the whole impossible puzzle.

Why This Matters for the Real World

In the past, economists often had to throw away data or make very strict, unrealistic assumptions (like "manufacturers are innocent and random") to study things like:

Mergers: What happens if two big companies combine?
Taxes: How much will sales drop if we tax soda?
Entry/Exit: What happens if a new competitor enters the market?

Because companies are strategic (they aren't innocent), those old methods often gave shaky results.

This paper says: "You don't need to assume companies are innocent. You just need to find a good 'price shock' (like a supply chain glitch) and use our 'Recentering' filter. You can get reliable answers about price changes even if the companies are playing games with their product features."

Summary

Old Way: "We can't solve this unless the product features are random." (Too strict, often impossible).
New Way: "We can solve this if we have a good price shock and if the product features are a 'faithful' clue to hidden demand."
The Tool: A mathematical filter called "Recentered Instruments" that strips away the strategic noise.
The Result: We can now trust our predictions about price changes (like tax hikes or merger effects) much more, even in messy, real-world markets where companies are strategic.

It's a bit like realizing you don't need to know the entire history of a crime to solve the specific question of "Who pulled the trigger?" You just need the right evidence and the right way to look at it.

1. Problem Statement

The paper addresses a fundamental challenge in the econometrics of differentiated product demand: nonparametric identification of price counterfactuals (e.g., merger simulations, welfare analysis) using market-level data when product characteristics are endogenous.

The Conventional Wisdom: Previous literature, notably Berry and Haile (2014), established that nonparametric identification of flexible demand models generally requires exogenous product characteristics ( $X$ ) to serve as instruments for market shares ( $S$ ). The intuition is that to separate the effects of prices ( $P$ ) from unobserved demand shocks ( $\xi$ ), one needs variation in $S$ that is independent of $P$ . If characteristics are endogenous (e.g., firms strategically choose features based on demand conditions), standard "BLP-style" instruments (functions of $X$ ) become invalid, and identification fails.
The Gap: The authors ask: Can price counterfactuals be identified if characteristics are endogenous, provided we have valid exogenous price instruments ( $Z$ )?
The Obstacle: Standard nonparametric instrumental variable (IV) theory suggests that price instruments alone are insufficient because they only shift prices, not shares directly. Without exogenous $X$ , the system is under-identified.

2. Methodology and Identification Strategy

The authors propose a novel identification strategy that relaxes the requirement for exogenous characteristics by introducing recentered instruments and a new technical condition called faithfulness.

A. Model Setup

The demand system is defined by:
$S = \sigma(\delta, \tilde{X}, P)$
$\delta = \delta(X, \xi)$
Where:

$S, P, X$ are vectors of shares, prices, and observed characteristics.
$\tilde{X}$ are "non-special" characteristics (fully conditioned on).
$\delta$ is an unobserved index combining demand shocks and characteristics.
Key Relaxation: Unlike Berry and Haile (2014), the authors do not assume $X$ is exogenous ( $X \perp \xi$ ). They only assume conditional exogeneity of price instruments: $Z \perp \delta \mid X$ .

B. Recentered Instruments

The authors utilize recentered instruments, defined as:
$R(X, Z) - E[R(X, Z) \mid X]$
These instruments are mean-zero conditional on $X$ . The core insight is that while $X$ may be endogenous, the variation in $Z$ that is orthogonal to $X$ provides identifying power. If a candidate inverse demand function $\check{\sigma}^{-1}$ is the true function, its implied index $\check{\delta}$ must be mean-independent of $Z$ given $X$ .

C. The Faithfulness Condition

The central theoretical contribution is the Faithfulness condition (Assumption 4).

Definition: The distribution of $(\delta, P) \mid (X, Z)$ is faithful if, for any function $H(\delta, P)$ , the condition $E[H(\delta, P) \mid X, Z] = k(X)$ (i.e., mean-independence from $Z$ given $X$ ) implies that $H$ does not depend on $P$ (i.e., $H(\delta, P) = H(\delta)$ ).
Intuition: Faithfulness ensures that the variation induced by $Z$ (conditional on $X$ ) is rich enough to detect any causal effect of prices on demand. If a candidate model survives the recentered instrument test (is mean-independent of $Z$ given $X$ ), faithfulness guarantees it is merely a transformation of the true demand function, not a spurious one.
Role of $X$ : In this framework, $X$ acts as a proxy for the unobserved index $\delta$ , not as an exogenous instrument. It need not be exogenous; it must simply be a "strong proxy" (highly predictive of $\delta$ ).

3. Key Results

A. Identification of Price Counterfactuals

Proposition 2: Under Assumptions 1 (invertibility), 3 (conditional exogeneity of $Z$ ), and 4 (Faithfulness), price counterfactuals are nonparametrically identified.

Mechanism: Any candidate $\check{\sigma}^{-1}$ in the identified set $\Theta_I$ must satisfy $E[\check{\sigma}^{-1}(S, P) \mid X, Z] = k(X)$ . Faithfulness implies $\check{\sigma}^{-1}(S, P) = H(\delta) = H(\sigma^{-1}(S, P))$ .
Invariance: Since price counterfactuals are calculated as $\sigma(H^{-1}(H(\delta)), p') = \sigma(\delta, p')$ , the unknown transformation $H$ cancels out. Thus, the counterfactual quantities are point-identified even if the demand function itself is only identified up to a transformation.

B. Relationship between Faithfulness and Completeness

The authors rigorously analyze the relationship between their new Faithfulness condition and the standard Completeness condition (Assumption 2).

Equivalence: In the limit where $Z = P$ (exogenous prices), Faithfulness and Completeness are equivalent (Proposition 3).
Sufficient Conditions for Faithfulness: The paper provides several non-nested conditions under which Completeness implies Faithfulness, showing that Faithfulness is not an "exotic" assumption but a natural consequence of standard conditions in many economic settings:
1. Price-Setting Restrictions: If prices depend on observables via a specific index structure (Assumption 5) or if the Jacobian of prices with respect to instruments satisfies a separability condition (Assumption 6). These are compatible with Bertrand-Nash pricing.
2. Statistical Restrictions: If $\delta$ and $X$ have finite support of the same size, or if $\delta \mid X$ follows a location-scale model with specific smoothness conditions (Assumption 7).
Distinctness: The authors provide counterexamples showing that Faithfulness can hold without Completeness (and vice versa), proving they are distinct but closely related concepts.

C. Exogenous Prices as a Baseline

Proposition 1: Even if all characteristics are fully endogenous, if prices are exogenous ( $P=Z$ ), price counterfactuals are identified under standard completeness. This highlights that the "2J" instrument requirement (J for prices, J for shares) in Berry and Haile (2014) is driven by the need to disentangle $P$ and $S$ effects. If $P$ is exogenous, we only need to detect the causal effect of $P$ , requiring only $J$ instruments.

4. Practical Implications

The paper offers significant guidance for applied researchers:

Robustness to Endogeneity: Researchers can estimate price counterfactuals even if product characteristics are endogenous (strategic firm choices), provided they have valid supply-side shocks ( $Z$ ).
Use of Recentered Instruments: Applied work should utilize recentered instruments (functions of $Z$ centered by $X$ ) rather than standard BLP instruments. This avoids bias from characteristic endogeneity and model misspecification.
Data Requirements: Identification is possible with market-level data only. Micro-data (consumer-level variation) is not strictly necessary, unlike in Berry and Haile (2024).
No Need for Exogenous Characteristics: Researchers do not need to justify that observed characteristics are "as-good-as-random." They only need to argue that $Z$ is exogenous conditional on $X$ and that $X$ is a strong proxy for demand heterogeneity.

5. Significance and Contribution

Theoretical Breakthrough: The paper overturns the "conventional wisdom" that exogenous characteristics are essential for nonparametric identification of demand. It demonstrates that price instruments + recentering + faithfulness are sufficient.
New Condition (Faithfulness): The introduction of the Faithfulness condition provides a new lens for nonparametric IV identification, linking it to causal discovery literature. It reframes identification as a problem of "detectability" of causal effects via variation strength rather than strict exclusion restrictions.
Policy Relevance: By relaxing the strict requirements on product characteristics, the paper enables more robust counterfactual analysis (e.g., merger simulations) in industries where firms strategically design products, a scenario where previous methods might have been biased or inapplicable.
Bridging Literature: It connects the demand estimation literature with nonparametric IV theory and triangular models, showing how results from one domain (e.g., Imbens and Newey, 2009) can be interpreted as verifying faithfulness.

In summary, Borusyak et al. provide a rigorous theoretical foundation for estimating demand counterfactuals in the presence of endogenous product characteristics, shifting the burden of identification from the exogeneity of characteristics to the strength and structure of price instruments and the use of recentered estimation techniques.