How Developers Adopt, Use, and Evolve CI/CD Caching: An Empirical Study on GitHub Actions
This paper presents the first empirical study of CI/CD caching on GitHub Actions, analyzing 952 repositories to reveal that cache-adopting projects are more active, utilize diverse and non-standardized mechanisms, and require significant ongoing maintenance driven by both human issue resolution and automated dependency updates.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are running a massive, high-speed bakery. Every morning, you have to bake hundreds of loaves of bread, but before you can bake, you have to mix the dough, preheat the ovens, and gather your ingredients.
If you did this from scratch every single time, it would take forever. So, you decide to use a smart shortcut: you keep a "ready-to-go" station where you store pre-mixed dough and pre-heated ovens. This is Caching. In the world of software (specifically on a platform called GitHub Actions), developers use caching to save time by reusing parts of their work instead of starting over every time they make a small change.
This paper is like a detective story where the authors went into 952 different "bakery kitchens" (software projects) to see how developers actually use, manage, and struggle with these shortcuts.
Here is what they found, broken down into simple stories:
1. Who Uses the Shortcuts?
The Analogy: Think of the "popular kids" at school.
The researchers found that the bakeries using these shortcuts were usually the busiest and most popular ones. They had more customers (stars), more bakers working together (contributors), and more orders coming in (commits).
- The Takeaway: If your project is small and quiet, you might not need the shortcut yet. But if you are a busy, popular project, you need caching to keep up with the demand.
2. How Do They Use the Shortcuts?
The Analogy: It's not just one magic button; it's a whole toolbox.
The authors expected everyone to use one standard way to save their dough. Instead, they found a chaotic mix of methods:
- The "Explicit" Method: Developers manually write down exactly what to save (like writing a specific note: "Save the chocolate chips"). This was the most popular method (70%).
- The "Automatic" Method: Some tools do it for them (like a robot arm that automatically saves the flour).
- The "Docker" Method: Saving the whole oven setup instead of just the ingredients.
- The Takeaway: There is no single "right" way. Developers are constantly experimenting with different ways to save time, and most prefer doing it manually because it gives them more control.
3. The "Tweak, Tweak, Tweak" Cycle (Evolution)
The Analogy: Imagine you set up your shortcut station, but the dough is slightly too sticky. You tweak the recipe. Then it's too dry. You tweak it again. Then you realize you forgot the salt. You tweak it again.
The study found that caching is never "set and forget." It is a living, breathing thing that requires constant babysitting.
- Build & Test Jobs: These are the high-pressure ovens. Developers here are constantly tweaking the settings (changing keys and paths) every few days. It's a cycle of "Try, Fail, Fix, Try Again."
- Release Jobs: These are the final packaging stations. Once they set the shortcut, they rarely touch it. It stays stable for months.
- The Takeaway: The most critical parts of the software (building and testing) require the most maintenance. It's not a one-time setup; it's a constant game of "tuning the engine."
4. Who is Doing the Work? (Humans vs. Robots)
The Analogy: Imagine a team of humans and a team of robots.
- The Robots (Bots): Their job is very specific. They mostly show up to say, "Hey, your shortcut tool is outdated! Update it to version 2.0!" They handle the boring version upgrades.
- The Humans: They do the heavy lifting. They are the ones saying, "The shortcut isn't working because the chocolate chips are in the wrong folder," or "We need to save the flour and the sugar now."
- The Takeaway: Robots handle the easy updates, but humans are stuck doing the hard work of fixing broken shortcuts and figuring out what to save.
5. The Hidden Cost
The Analogy: You save time baking, but you spend time cleaning the shortcut station.
The paper reveals a surprising truth: Caching is actually a lot of work.
Developers spend a huge amount of time fixing their caching configurations. They are constantly removing old shortcuts that don't work and adding new ones. Sometimes they add a shortcut, realize it's useless, and remove it the next day.
- The Takeaway: Caching saves you time on running the software, but it costs you time on managing the software. It's a trade-off.
The Big Lesson for Everyone
The authors conclude that we need to stop treating caching like a simple "on/off switch." It's more like a gardener's tool. You plant the seeds (enable caching), but then you have to water, weed, and prune it constantly (maintenance).
What should we do?
We need better tools (maybe smarter robots) that can help humans fix these shortcuts automatically. Right now, humans are doing too much of the "weeding" manually, and it's slowing them down. If we had better tools, developers could spend less time fixing their shortcuts and more time baking the bread (writing code).
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.