Measuring the Invisible: Evaluating the Impact of Public Funding on Open Source Software
This study employs a Generalized Synthetic Control Method to demonstrate that the Sovereign Tech Fund significantly increases open source project activity metrics like commits and pull requests by mobilizing existing development efforts, rather than expanding the contributor base or accelerating backlog resolution.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Picture: The Invisible Foundation
Imagine the digital world (apps, websites, banking systems) as a massive skyscraper. Most people see the shiny glass and the people inside, but they don't see the steel beams, the plumbing, or the electrical wiring holding it all up.
Open Source Software (OSS) is that hidden steel and plumbing. It's code that anyone can see and use for free. It runs almost everything we use today. However, because it's free and built by volunteers, it often suffers from a "free rider" problem: everyone uses it, but no one pays to fix the leaks or reinforce the beams. When a critical piece breaks (like the famous log4j vulnerability), the whole building is at risk.
To fix this, the German government created the Sovereign Tech Agency (STA). Think of them as a specialized construction fund that gives money to the teams maintaining these critical digital "beams" so they don't collapse.
The Question: Does the Money Actually Work?
The author, Laia Domenech Burin, asked a simple but hard question: Does giving this public money actually make the software better, or is it just a waste of tax dollars?
It's hard to answer because you can't run a "lab experiment" on the internet. You can't take two identical software projects, give money to one, and watch the other starve to see what happens.
The Solution: Building a "Ghost Twin"
To solve this, the author used a clever statistical trick called the Generalized Synthetic Control Method (GSCM).
Imagine you want to know if a new fertilizer makes a specific tomato plant grow faster. You can't just compare it to a random plant in a different garden. Instead, you build a "Ghost Twin" of your tomato plant.
- You look at hundreds of other tomato plants that didn't get the fertilizer.
- You mix them together in a specific recipe (a weighted average) to create a "Ghost Twin" that looks, acts, and grows exactly like your real plant before you gave it the fertilizer.
- Once you give the real plant the fertilizer, you watch both the real plant and the Ghost Twin.
- If the real plant grows taller than the Ghost Twin, you know the fertilizer worked.
In this paper, the "tomato plants" are critical software projects (like PyPI for Python, curl for data transfer, Fortran for science, and RubyGems). The "Ghost Twins" were built using data from 62 other similar software projects that didn't get the money.
What They Found: The "Speed" vs. The "Size"
The study looked at four main things to measure success:
- Commits: How often code is updated.
- Pull Requests: New ideas or fixes being proposed.
- Issues: Bugs being reported or new features requested.
- Contributors: New people joining the team.
The Results:
- The Good News: The money acted like a turbocharger. The funded projects started working much faster. They had significantly more code updates, more new ideas being proposed, and more bugs being reported. The "Ghost Twins" stayed the same, but the funded projects sped up.
- The Reality Check: The money did not act like a magnet. It did not bring in new people to the team (the number of contributors didn't change). It also didn't necessarily clear out the old backlog of bugs faster (the number of "closed issues" didn't go up significantly compared to the Ghost Twin).
The Analogy: The Busy Workshop
Think of the software project as a busy workshop.
- Without funding: The workers are tired, working on old tools, and barely keeping up.
- With funding: The workers get better tools and a salary.
- Result: They start working much faster and tackling more tasks immediately (more commits and new requests).
- But: They don't suddenly hire a bunch of new apprentices (no new contributors), and they don't necessarily finish the pile of old, dusty work faster than they would have anyway (no massive drop in closed issues).
The Conclusion
The paper concludes that public funding is effective at mobilizing existing energy. It wakes up the current team and gets them moving faster. However, it doesn't automatically solve the deeper problems of finding more people to help or clearing out years of old debt.
The author suggests that if a government wants to fix a specific problem (like "we need more people" or "we need to clear the backlog"), they need to design their funding programs specifically for that goal, rather than expecting one type of money to fix everything at once.
In short: The money works, but it works like a speed boost for the current team, not a recruitment drive or a backlog cleaner.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.