📈 economics

Diagnostic Feedback under Hidden Task Difficulty

This paper demonstrates that when task difficulty is privately observed by an evaluator, the strategic decision to purchase an ability diagnostic can uniquely reveal the hidden difficulty level and shape agent effort, a phenomenon that does not occur when difficulty is public or adoption is independent of it.

Original authors: Mark Izgarshev, Georgy Lukyanov

Published 2026-07-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Mark Izgarshev, Georgy Lukyanov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are standing in a room where people are about to take a test, but nobody knows exactly how hard the test is or how smart the people taking it are. In the world of economics, this is a classic puzzle about "signaling." Usually, we think of a signal as a message sent to show off something you already know, like a peacock spreading its tail to show it's healthy. But sometimes, the act of sending a signal tells a story that the message itself doesn't. Think of it like this: if you see someone buying a very expensive, high-tech map before a hike, you might not know the terrain just from the map, but you know they think the hike is dangerous. If the hike were a easy stroll, buying that map would be a waste of money. So, the purchase itself reveals the danger. This paper dives into a specific corner of game theory—a branch of math that studies how people make decisions when they are trying to guess what others are thinking. It asks a tricky question: Can the decision to get information actually create more information than the information itself? It turns out that hiding the truth about a situation can sometimes force people to reveal the truth about themselves.

The authors of this paper, Mark Izgarshev and Georgy Lukyanov, set up a story with two main characters: a "Manager" (the evaluator) and a "Worker" (the agent). The Manager knows if the upcoming job is a "Cake Walk" (easy) or a "Mountain Climb" (difficult), but the Worker doesn't. Neither of them knows if the Worker is naturally "Super Fit" (high ability) or "Average" (low ability). Before the Worker starts, the Manager can choose to buy a "Diagnostic Test" (like a practice exam or a skills audit) that reveals the Worker's true fitness level. This test costs money.

Here is the twist: The test is only useful for specific combinations. If the job is a Cake Walk, the "Average" worker is the one who needs to try hard to succeed, while the "Super Fit" worker is already good enough that trying harder doesn't help much. But if the job is a Mountain Climb, the "Super Fit" worker is the one who needs to push hard, while the "Average" worker would struggle no matter what. The test tells the Worker their level, which then tells them whether they should sweat and work hard or just chill.

The paper finds a surprising result: When the Manager keeps the difficulty of the job a secret, the "Mountain Climb" Manager will buy the test, but the "Cake Walk" Manager will not. Why? Because if the Manager buys the test, the Worker realizes, "Oh, they only bought this test because the job is hard!" Once the Worker knows the job is hard, the "Super Fit" Worker gets motivated to work hard (because the test told them they are fit enough to handle it), while the "Average" Worker gives up. This is great for the Manager of the hard job. However, if the Manager of the easy job tried to copy this and buy the test, it would backfire. The test would reveal that the Worker is "Average," and since the job is easy, the "Average" Worker would be the one motivated to work. But the test costs money, and the extra work isn't worth the price tag for an easy job. So, the easy Manager refuses to buy the test.

The paper proves that while a scenario where neither manager buys the test is mathematically possible, it relies on the Worker making an unrealistic guess that a test purchase must come from the easy manager (who would never want one). If we rule out such unreasonable guesses, the "separating" outcome—where only the hard-job manager buys the test—is the unique logical result. If the Manager didn't buy the test, the Worker would assume the job is easy and do nothing. If the Manager did buy the test, the Worker knows it's a hard job. The test purchase itself becomes a signal that screams "This is a hard job!"

The authors also show that this magic only happens because the difficulty is hidden. If the Manager simply announced, "This is a hard job!" without buying a test, the "Cake Walk" Manager would lie and say, "No, it's hard!" just to get the Worker to try harder, because lying is free. But buying a test costs money, so only the Manager who really needs the Worker to try hard (the one with the hard job) is willing to pay for it.

Interestingly, the paper shows that whether this outcome is better or worse for society is not a settled question; the model does not establish a clear welfare ranking. Sometimes, if the difficulty is public, the "Average" workers on hard jobs might try anyway, which could be good for the total amount of work done. But when the difficulty is hidden, the system filters out the "Average" workers on hard jobs and only the "Super Fit" ones work. The paper doesn't say one way is morally better, but it shows that hiding the difficulty of a task can actually generate more useful information about a person's ability than if the difficulty were known from the start. It's a counter-intuitive idea: sometimes, keeping a secret about the world forces people to reveal secrets about themselves.

The authors tested this with different scenarios, like what happens if the test isn't perfect (it makes mistakes) or if the Worker can choose to work a little bit or a lot instead of just "all or nothing." In all these cases, the main story holds up: as long as the test is reasonably accurate and the cost is in the right range, the hidden difficulty drives the Manager to buy the test, and the test purchase tells the Worker exactly how hard the job is. The paper concludes that this mechanism is robust, meaning it's not just a fluke of a simple math model, but a real possibility in how information and motivation interact in the real world.

Technical Summary: Diagnostic Feedback under Hidden Task Difficulty

Problem and Motivation
The paper investigates a signaling game where an informed evaluator (the principal) decides whether to purchase a diagnostic tool to reveal an agent's ability before the agent exerts costly effort. The core tension arises because the diagnostic serves a dual purpose: it provides the agent with self-knowledge (ability), but the decision to purchase it (adoption) may reveal the evaluator's private information regarding the task's difficulty.

The authors challenge the intuition that concealing information about the task should reduce or leave unchanged the information produced about the agent. Instead, they demonstrate that hidden task difficulty can generate information about the agent that would not exist under transparency. This occurs because the adoption decision acts as a second signal, pointing to the task rather than the person, which alters how the agent interprets the diagnostic result.

Methodology and Model Setup
The authors employ a game-theoretic framework with two players: an evaluator ( $P$ ) and an agent ( $A$ ).

Environment: Nature draws a task difficulty $t \in \{E, D\}$ (Easy or Difficult) and an agent ability $\theta \in \{L, H\}$ (Low or High). The evaluator observes $t$ ; the agent observes neither $t$ nor $\theta$ initially.
Effort and Returns: The agent chooses binary effort $e \in \{0, 1\}$ at cost $c$ . The marginal return to effort crosses across task-ability pairs: effort is most productive for Low ability on an Easy task and for High ability on a Difficult task. This "crossing" property is derived from a standard threshold success technology where effort matters most for the agent closest to the success requirement.
Diagnostic: The evaluator can publicly purchase a diagnostic ( $z=1$ ) at cost $K$ that perfectly reveals $\theta$ . If $z=0$ , ability remains unknown.
Timing: Nature draws $(t, \theta)$ ; $P$ observes $t$ ; $P$ chooses $z$ ; if $z=1$ , $\theta$ is revealed; $A$ chooses $e$ ; success is realized.
Assumptions: The analysis focuses on a parameter region where High ability is common ( $\pi > 1/2$ ), Difficult tasks are rare ( $\rho$ is small), and the diagnostic cost $K$ is intermediate. Crucially, without a diagnostic, the agent does not work on Easy tasks (expected return $< c$ ) but works on Difficult tasks (expected return $> c$ ) due to the pooling of ability types.

Key Results

Feedback Reversal: The motivational effect of a diagnostic result depends on the task.
- On an Easy task, a Low-ability agent works (because the result is "unfavorable" relative to the requirement, making effort pivotal), while a High-ability agent does not.
- On a Difficult task, a High-ability agent works (favorable result), while a Low-ability agent does not.
- Thus, "favorable" feedback induces effort only when adoption reveals a Difficult task; "unfavorable" feedback induces effort only when adoption reveals an Easy task.
Separating Equilibrium under Hidden Difficulty:
- Under hidden difficulty, a unique Perfect Bayesian Equilibrium (PBE) satisfying the D1 refinement exists where only the Difficult-task evaluator purchases the diagnostic.
- Mechanism: The Difficult evaluator buys the diagnostic to separate herself from the Easy evaluator. If she does not, the agent assumes the task is Easy (or pools on no diagnosis) and does not work. By buying, she signals a Difficult task; the agent then works if the result is High ability. The Easy evaluator cannot mimic this because the cost $K$ exceeds the expected gain from inducing Low-ability agents to work on an Easy task.
- D1 Selection: Pooling equilibria (where neither buys) are eliminated by D1 because an unexpected diagnostic is strictly unprofitable for the Easy type under any agent response, but profitable for the Difficult type if the agent infers a Difficult task.
Benchmarks (Transparency and Commitment):
- Public Difficulty: If task difficulty is public, neither evaluator diagnoses.
  - On Easy tasks: The diagnostic motivates too few agents (only Low ability) to cover the cost.
  - On Difficult tasks: The diagnostic is counter-productive. Without it, the uncertainty about ability motivates both types to work (since the expected return is high). Revealing ability stops Low-ability agents from working, reducing the evaluator's payoff.
- State-Independent Commitment: If the evaluator must commit to a policy before observing difficulty, she chooses no diagnostic. The instrument's value lies in its ability to signal the specific state (task difficulty); without this state-contingency, the adoption decision carries no information, and the diagnostic is not worth the cost.
Robustness:
- The mechanism survives continuous effort (where effort intensity varies with perceived productivity) and imperfect diagnostics (noisy signals), provided the diagnostic accuracy is sufficiently high.
- The results reverse exactly if the modal ability type is Low and tasks are Easy, confirming the mechanism relies on the crossing returns and the rarity of the "productive" task-ability pair, not on specific labels.

Significance and Contributions
The paper makes three primary contributions to the literature on signaling, information design, and motivation:

Endogenous Information Acquisition as Signaling: Unlike standard models where the principal's information is about the agent, here the principal's private information is about the environment (the task). The paper shows that the act of acquiring information about the agent (the diagnostic) can serve as a credible signal about the environment. This creates a wedge between hidden and public information regimes that does not exist in standard cheap-talk models.
Diagnostic Over-Adoption: The authors identify a phenomenon where assessments are purchased with positive probability under hidden difficulty but with zero probability under transparency. This "over-adoption" is not a value-of-information effect but a signaling equilibrium where the purchase itself is the valuable signal.
Context-Dependent Feedback: The paper demonstrates that the motivational impact of feedback (favorable vs. unfavorable) is not intrinsic to the label but is determined by the inference drawn from the adoption of the feedback mechanism. This provides a theoretical basis for heterogeneous and sign-reversing effects of feedback observed in empirical studies.

Welfare Implications
The paper explicitly states that diagnostic over-adoption does not automatically imply a welfare gain.

Hidden difficulty improves welfare relative to uninformative pooling by generating productive effort from High-ability agents on Difficult tasks.
However, compared to public difficulty, the welfare ranking is ambiguous. Public difficulty saves the diagnostic fee and may sustain effort from Low-ability agents on Difficult tasks (due to unresolved uncertainty). Whether concealing difficulty is socially preferable depends on the social value of the extra Low-ability effort versus the cost of the diagnostic fee.

Conclusion
The paper concludes that concealing difficulty can generate more information about the person because the act of seeking it becomes informative about the task. The mechanism relies on crossing returns to effort and the inability of the evaluator to commit to a state-independent policy. The findings suggest that in settings where evaluators privately know task difficulty, the decision to assess is as informative as the assessment itself.

Technical Summary: Diagnostic Feedback under Hidden Task Difficulty

More like this