📈 economics

Targeting Efficiency in Means Testing: A Comparison of Money-Metric and Counting Approaches

This paper demonstrates that when using identical household data, a money-metric aggregation approach significantly outperforms a counting-based approach in targeting efficiency for means testing, establishing aggregation structure as a critical determinant of social protection performance.

Original authors: Federico Perali, Martina Menon, Eva Sierminska

Published 2026-08-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Federico Perali, Martina Menon, Eva Sierminska

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the head chef of a massive soup kitchen trying to feed the hungriest people in town. You have a list of ingredients (money, savings, health, family size) that might tell you who is truly starving. But here's the tricky part: how do you mix those ingredients to decide who gets a bowl? Do you add up the value of everything they have, like a financial calculator? Or do you just count how many "bad things" are on their list, like a checklist of misfortunes? This is the heart of a field called means testing, which is the process governments use to figure out who deserves help. The big question isn't just what information we collect, but how we combine it. If we mix the ingredients wrong, we might accidentally feed the people who have full bellies while leaving the truly hungry ones waiting. This paper dives into that exact problem, comparing two different "recipes" for deciding who gets the soup.

The researchers, Federico Perali, Martina Menon, and Eva Sierminska, set up a fascinating experiment to see which recipe works better. They didn't just guess; they used real data from 7,951 Italian households to test two distinct approaches. The first approach is the Money-Metric Index. Think of this like a sophisticated smoothie blender. You throw in income, savings, health, and family size, and the blender mixes them all together into a single, smooth number. It understands that a dollar is worth more to a single person than to a family of five, and it can trade off a little bit of bad health against a lot of savings. It's a "cardinal" approach, meaning it treats well-being as a continuous scale where things can cancel each other out.

The second approach is the Counting Method. Imagine this as a "bad luck bingo" card. Instead of blending everything, you just count how many boxes you can check off. Did you have low income? Check. Do you have no savings? Check. Is someone sick? Check. If you check off enough boxes (say, three out of five), you are declared poor. This method treats every box as a separate, unchangeable fact. It doesn't let your savings "compensate" for your bad health; it just counts the hits.

The paper puts these two methods through a rigorous stress test. They fed both methods the exact same information about the same families—same income, same assets, same health issues, same family sizes. The only thing that changed was the "recipe" for combining the data. To see who was right, they compared the results against a "benchmark" group of people who were genuinely struggling (defined by low income, inability to save, and self-reported hardship).

The results were clear and surprising, though with an important nuance. The Money-Metric approach (the blender) was generally better at finding the right people, but it wasn't perfect. It did make mistakes, specifically "Type II errors," meaning it did leave out a significant number of people who actually needed help (about 38% in some tests). However, the Counting approach (the bingo card) was much more finicky and often worse at this specific task. As the researchers added more categories to the bingo card (like adding "unemployment" or "retirement" to the list), the counting method started missing a huge number of poor people, excluding almost everyone who didn't fit a very specific, narrow profile of misery.

In one specific scenario, the two methods were statistically tied in their total number of mistakes. But even in that tie, they made different kinds of mistakes. The money-metric method was much better at avoiding "Type I errors" (giving soup to people who didn't need it), while the counting method was much worse at it. When the researchers weighed both types of mistakes equally, the money-metric approach generally offered a more balanced and reliable performance, whereas the counting approach became extremely sensitive to how many boxes were on the card.

One of the most interesting findings was how the two methods viewed retirement. The counting method treated being retired as a "bad box" to check off, automatically flagging many retired people as poor, even if their pension was enough to live on. The money-metric method, however, looked at the actual money coming in from that pension and decided: "Okay, they have enough income, they aren't poor." The counting method was essentially judging people by their label (retired), while the money-metric method judged them by their actual situation.

The authors suggest that the counting method is too rigid. It's like trying to judge a complex movie by only counting how many times the hero cries; you might miss the whole story. The money-metric method, by blending everything together, creates a more accurate picture of a household's real life. While the counting method can be simpler to administer, the paper shows that it leads to much higher "exclusion errors"—leaving the truly needy out in the cold—or forces a trade-off where you must let many ineligible people in to catch the poor.

In the end, the study suggests that if we want to build fair and efficient social safety nets, especially in places like the European Union where countries are trying to coordinate their systems, we should lean toward the "blender" approach. It's not just about having more data; it's about having a smarter way to mix it. The money-metric approach doesn't need secret information that the counting method doesn't have; it just processes the shared information in a way that respects the complexity of real life, ensuring that the people who are truly struggling get the help they need with fewer overall errors.

Technical Summary: Targeting Efficiency in Means Testing

Problem Statement
The effectiveness of means-testing instruments for social protection depends on two critical design choices: the selection of information to collect about households and the method used to aggregate that information into eligibility criteria. While the literature has extensively debated the relative merits of different informational bases (e.g., income vs. assets vs. multidimensional indicators), the second choice—how dimensions of deprivation are combined—has received comparatively less scrutiny. This paper addresses this gap by investigating whether different aggregation structures, applied to an identical informational base, yield systematically different targeting outcomes. Specifically, it compares a money-metric index, which aggregates dimensions cardinally based on welfare theory, against a counting-based poverty measure, which treats dimensions as binary and non-compensable. The central question is whether these two approaches are interchangeable tools for identifying eligible beneficiaries or if the aggregation structure itself is a primary determinant of targeting efficiency.

Methodology
The study employs a controlled experimental design using household-level data from the 2010 Italian Survey of Household Income and Wealth (SHIW), conducted by the Bank of Italy. The sample comprises 7,951 households.

Informational Base: Both approaches utilize an identical set of dimensions: disposable income ( $Y$ ), financial wealth ($FW$), non-financial wealth ($NFW$), and four normative household characteristics: poor health ($HE$), single parenthood ($SP$), unemployment ( $U$ ), and retirement ($RE$).
The Money-Metric Approach: This method aggregates dimensions into a single cardinal welfare indicator. It converts net worth into an annuity value and combines it with disposable income. Crucially, it applies equivalence scales (both data-driven and hybrid) to adjust for household composition and normative characteristics (e.g., disability, single parenthood). This ensures interpersonal comparability and allows for compensation across dimensions (e.g., high assets can offset low income). The resulting index is Cardinally Fully Comparable (CFC).
The Counting Approach: Based on the Alkire-Foster (2011) methodology, this approach identifies households as deprived dimension-by-dimension relative to specific cutoffs. It constructs a deprivation matrix where households are classified as poor if their weighted deprivation count exceeds a threshold ( $K$ ). In this framework, dimensions are treated as binary; there is no cardinal aggregation prior to identification, and no interpersonal welfare ranking is generated.
Benchmark Construction: To evaluate targeting performance, the authors construct an exogenous "benchmark poor" group. A household is included in this benchmark only if it satisfies three conditions simultaneously: (1) low equivalent disposable labor income (below the 40th percentile), (2) low saving capacity (unable to save or drawing down savings), and (3) self-reported economic hardship ("with many difficulties").
Performance Metrics: The study measures Type I errors (inclusion errors: non-poor households incorrectly classified as eligible) and Type II errors (exclusion errors: poor households incorrectly denied support). Targeting efficiency is defined as the method that minimizes the joint incidence of these two errors, assigning equal weight to both.

Key Contributions

Unified Framework for Comparison: The paper develops a transparent framework that isolates the pure effect of aggregation structure on targeting performance. By holding the informational content, deprivation thresholds, and equivalence scales constant across both methods, the study attributes differences in outcomes solely to the aggregation rule (cardinal vs. binary/non-compensable).
Empirical Assessment of Targeting Errors: The authors provide a direct empirical measure of policy performance by estimating inclusion and exclusion errors under alternative eligibility thresholds and dimension counts, moving beyond theoretical comparisons to practical targeting efficiency.
Implications for Policy Harmonization: The study documents the consequences of these methodological differences for the harmonization of social protection eligibility criteria, particularly within the European Union context where systems currently rely on disparate approaches.

Results
The empirical analysis yields several robust findings:

Superiority of the Money-Metric Approach: The money-metric approach consistently delivers substantially lower total targeting errors than the counting approach. It achieves a better balance between inclusion and exclusion errors across various specifications.
Sensitivity to Dimensionality: The counting approach is highly sensitive to the number of dimensions considered. As the number of dimensions increases, the counting method's ability to correctly identify the poor declines sharply (increasing exclusion errors), while inclusion errors may drop to near zero. In contrast, the money-metric approach remains more stable regardless of dimensionality.
Role of Assets: Incorporating asset information (financial and non-financial) significantly reduces inclusion errors for both methods but is particularly effective in the money-metric framework. Ignoring assets leads to systematic misclassification and higher leakage.
Treatment of Heterogeneity: The counting approach treats normative characteristics (like retirement or single parenthood) as explicit deprivation dimensions, effectively categorizing households based on status regardless of their actual resource levels. The money-metric approach integrates these characteristics through equivalence scales, evaluating the outcome of these statuses on welfare. Consequently, the two methods identify markedly different profiles of "poor" households; for instance, the counting method identifies a much higher proportion of retired households as poor compared to the money-metric method, which accounts for pension income.
Statistical Significance: The difference in total error rates between the two approaches is statistically significant (exceeding eight standard errors in most specifications), confirming that the aggregation structure is a first-order determinant of performance.

Significance and Claims
The paper claims that aggregation structure is a first-order determinant of means-testing performance, independent of the informational content used. The findings suggest that the money-metric approach offers a more reliable and theory-consistent basis for constructing unified eligibility criteria.

The authors argue that the counting approach's "atheoretical" aggregation of binary deprivations across incommensurable dimensions fails to provide a complete interpersonal ordering of well-being, leading to high exclusion errors and unstable targeting as dimensions are added. Conversely, the money-metric approach, grounded in welfare theory and equivalence scales, preserves interpersonal comparability and allows for compensatory trade-offs between dimensions.

For policymakers, particularly in the context of modernizing and harmonizing social protection systems (e.g., within the EU), the paper concludes that the choice of aggregation structure is not neutral. A money-metric instrument does not necessarily require superior data to outperform counting methods; rather, it processes shared information more effectively by respecting the cardinal nature of welfare and the heterogeneity of household needs. This structural advantage makes it a more coherent tool for minimizing targeting errors and ensuring equitable access to social transfers.

Technical Summary: Targeting Efficiency in Means Testing

More like this