Software is infrastructure: failures, successes, costs, and the case for formal verification
This chapter argues that because software functions as critical infrastructure and the staggering costs of historical failures demonstrate the severe consequences of poor quality, the adoption of formal verification and program analysis is essential, a stance supported by successful industrial applications.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The Big Idea: Software is the New Concrete
Imagine a world where our roads, bridges, and power plants are made not of steel and concrete, but of invisible code. The authors argue that software has become the infrastructure of modern society. Just as a bridge needs to hold up a truck without collapsing, our software (which runs hospitals, banks, planes, and even your toaster) needs to work perfectly.
The paper asks a simple but scary question: If a bridge is built with bad math, it falls down. If software is built with bad math, what happens? The answer is: billions of dollars vanish, people get hurt, and sometimes, people die.
The Problem: We Are Building Castles in the Sand
The authors point out that we treat software differently than physical engineering.
- Building a Wall: If you build a wall, physics does the testing. If the wall is too weak, gravity knocks it down before you even paint it. You can't "run" a wall to see if it works; you just build it and hope the math holds.
- Writing Software: Software is just text. You can't "feel" a bug. You have to run the code to see if it works. But running the code is like driving a car off a cliff to see if the parachute opens. By the time you find the bug, the crash has already happened.
The paper uses a funny example: If you type rm -rf ~ in a computer terminal, it deletes your entire home folder. You don't need to run it to know it's dangerous; you just need to read the manual (the "math") to understand what it does. But for complex code, reading the manual isn't enough.
The Cost of "Bad Math": A Trillion-Dollar Leak
The paper lists a "Hall of Shame" of software failures over the last 40 years to show how expensive mistakes are. Think of these as the "bridge collapses" of the digital world:
- The Therac-25 (Healthcare): A radiation machine gave patients massive overdoses because the code let two buttons be pressed too fast. Result: 6 deaths.
- The London Ambulance (Emergency Services): A new dispatch system had a memory leak (like a bucket with a hole in it). It filled up with old data and crashed. Result: Ambulances couldn't find patients; 20–30 people died.
- The Boeing 737 MAX (Aviation): A software system called MCAS pushed the plane's nose down based on a single faulty sensor. Result: Two crashes, 346 deaths, and $20 billion in costs.
- The Horizon Scandal (Banking): A faulty accounting system told thousands of shopkeepers they were stealing money. Result: 900+ people were wrongly jailed, and the system cost taxpayers over £1 billion to fix.
- CrowdStrike (Global IT): A tiny update error caused millions of computers worldwide to turn blue and die. Result: Global chaos, costing billions in lost business.
The authors calculate that poor software quality costs the US economy $1.56 trillion a year. That is more than the entire GDP of many countries. It is money purely wasted on fixing mistakes that could have been prevented.
The Solution: The "Mathematical Blueprint"
The paper argues that we need to stop guessing and start proving our software works before we run it. This is called Formal Verification.
The Analogy:
Imagine you are building a skyscraper.
- Current Method (Testing): You build the 100th floor, then the 101st, then the 102nd. You check if the elevator works. If the 102nd floor collapses, you tear it down and try again. This is expensive and dangerous.
- Formal Verification: Before you pour a single drop of concrete, you use advanced mathematics to prove that the design cannot collapse under any weight. You check the blueprint against the laws of physics to ensure it is perfect.
In software, this means using math to prove that the code will do exactly what it is supposed to do, and nothing else.
Does It Pay Off? Yes, It's a Bargain
You might think, "Math is hard and expensive. Is it worth it?" The paper says yes, absolutely.
- **Air
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.