Correcting Growth-Based Benchmarking Bias: Time-Comparable Efficiency Levels and Context-Sensitive Pathways to High Performance
This study addresses growth-based benchmarking bias in public services by introducing a time-comparable efficiency framework that distinguishes between productivity growth and stable performance levels, revealing that rapid growth often masks low baselines while identifying four distinct, context-sensitive pathways to sustained high efficiency in China's provincial fitness services.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Problem: The "Fastest Runner" Trap
Imagine you are a coach trying to decide which athlete is the "best" to train for the Olympics. You have two runners:
- Runner A starts the race running at a snail's pace but suddenly sprints incredibly fast, doubling their speed in one minute.
- Runner B starts at a world-class speed and maintains that steady, high speed the whole time, maybe getting just a tiny bit faster.
If you only look at who improved the fastest, you would pick Runner A. But if you look at who is actually the fastest right now, it's clearly Runner B.
This paper argues that many governments make the same mistake. They look at public services (like community fitness programs) and praise the places that are "growing the fastest." The authors call this the "Growth Illusion." They say that just because a place is improving quickly doesn't mean it's already good; it might just be starting from a very low baseline. Meanwhile, the places that are already excellent might look "boring" because they aren't changing as dramatically, even though they are the true models of success.
The Solution: A Better Stopwatch and a New Map
To fix this, the authors used two special tools to study fitness services across 31 Chinese provinces from 2016 to 2022.
1. The "Unchanging Ruler" (Window-DEA)
Standard ways of measuring efficiency are like using a different ruler every year. If the ruler changes size, you can't compare how tall someone was last year versus this year.
The authors used a method called Window-DEA. Think of this as using a single, unchanging ruler that looks at a three-year "window" of time at once. This lets them compare the actual level of performance across different years fairly, without getting tricked by the "fast growth" of places that were previously very inefficient.
2. The "Recipe Book" (Panel QCA)
Once they knew who the actual high performers were, they asked: "What is the secret recipe for success?"
They didn't just look for one single factor (like "more money = better results"). Instead, they used a method called QCA (Qualitative Comparative Analysis). Think of this like a chef realizing there isn't just one way to make a great cake. You can make a great cake with:
- Lots of sugar and no eggs.
- No sugar but lots of eggs and chocolate.
- A mix of fruit and nuts.
The study found that there are four different "recipes" (pathways) that lead to high efficiency in fitness services. There is no single "best practice" that works for everyone.
The Four "Recipes" for Success
The study found that different provinces succeeded by mixing and matching five ingredients: Politics (government attention), Economy (money), Culture (education), Society (safety nets/tech), and Ecology (air quality/nature).
Here are the four ways they found to get a high score:
- The "Nature & Wallet" Route: Some places didn't need heavy government rules or highly educated populations. Because they had rich citizens and great air/nature, people just naturally exercised outdoors. The market and the environment did the work.
- The "Rich & Independent" Route: In very wealthy cities, people had so much disposable income that they could buy their own fitness gear and gym memberships. Even if the government's social safety net was weak, the residents' money kept the system running efficiently.
- The "Green & Safe" Route: In poorer areas where people didn't have much money, success came from great air quality and a strong social safety net (like good health insurance). People felt safe and healthy enough to exercise outdoors, compensating for the lack of cash.
- The "Government & Culture" Route: This was the most common "safety net" recipe. Here, strong government leadership and social security were the core ingredients. Whether the place was rich or poor, if the government steered the ship well and protected the people, they could find a way to succeed (sometimes adding nature, sometimes adding money).
The "Growth Illusion" in Action
The paper gives a specific example of Tibet.
- The Illusion: Tibet showed massive "growth" numbers. It looked like a superstar because it was improving so fast.
- The Reality: When the authors used their "unchanging ruler," they saw that Tibet was still starting from a very low base. It was a "catch-up" story, not a "high-performance" story.
- The Contrast: Places like Shanghai and Beijing didn't show huge growth spikes because they were already near the top. But they were the actual high performers. If you had only looked at growth, you would have ignored them.
The Takeaway
The main lesson is that speed of improvement is not the same as quality of performance.
- Don't just reward the "comeback kids." If you want to know who to copy for policy, look at who is consistently good, not just who is changing the fastest.
- There is no one-size-fits-all. A rich city doesn't need to copy a poor village's strategy. A place with great air doesn't need to copy a place with great technology.
- Fit matters more than size. Success comes from matching your local conditions (money, nature, rules) with the right strategy, rather than just throwing more resources at the problem.
In short: Stop measuring how fast a car is accelerating and start measuring how fast it is actually driving. And remember, different cars need different fuel to win the race.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.