SPECMINE: A Large-Scale Corpus of Spec-Driven Development Artifacts

In the world of software creation, there has long been a fundamental tension between the idea of a program and the program itself. Traditionally, a developer would hold a vision in their mind or scribble notes on a napkin, then spend months translating that vision into lines of code. The code was the primary artifact, the thing that was written, reviewed, and maintained. However, a new practice has recently emerged where the order is flipped. Before a single line of code is written, a developer, often with the help of an artificial intelligence tool, drafts a detailed, structured description of what needs to be built. This description, written in plain language, acts as a blueprint. An AI agent then reads this blueprint and generates the actual software. This shift means that the most important document in the project is no longer the code, but the specification itself. Understanding how these specifications are written, how they change, and how they eventually become working software is now a critical question for anyone studying how technology is made.

Until now, no one had a clear map of this new landscape. While dozens of tools have appeared to help developers write these blueprints, the actual documents they produce have never been studied on a large scale. Researchers Shyam Agarwal and Bogdan Vasilescu from Carnegie Mellon University have changed that by creating SPECMINE, a massive collection of these specification documents and the code they produce. They gathered nearly half a million specification files from public projects on GitHub, capturing a snapshot of this practice as it was born in 2025 and 2026. This collection includes not just the text of the specifications, but also the full history of the projects they belong to, the people who wrote them, and the specific changes made to the code when the specifications were updated. It is the first time the software community has been able to see, in detail, how a written description transforms into a functioning program in the age of AI.

The researchers built this collection in two main ways to ensure they captured the full picture. First, they looked for files named "spec.md" or "specs.md," which is the standard name used by most of the new tools. This search alone recovered over 470,000 files from more than 73,000 different repositories. They then added a second, separate search for a specific tool called AWS Kiro, which uses a different naming convention with separate files for requirements, design, and tasks. This added another 98,000 files. By combining these searches, they created a broad census that covers the vast majority of the tools currently in use. Every file in this collection is tagged with information about the project it came from, such as how many people have watched the project, what programming language it uses, and which specific tool generated the specification.

To understand how these written plans actually lead to code, the researchers went a step further. They focused on a smaller, carefully selected group of popular projects and tracked every request to change the code that involved a specification file. They found nearly 6,000 such requests, known as pull requests, where a developer or an AI agent updated the blueprint and the code at the same time. This allowed them to see the direct link between the idea and the implementation. In many cases, the specification and the code changed together in a single update, confirming that the blueprint was being used to drive the work in real time. However, the researchers also noted that this is not the only way it happens; sometimes a plan is written first and the code comes later, or the code is written first and the plan is added afterward. The data shows that while the simultaneous change is common, the relationship is complex and varies from project to project.

Beyond just counting files, the team analyzed the structure of these documents to see what they actually look like. They found that the specifications are not just random notes; they often follow specific patterns, such as using clear headings, lists, or structured formats that make them easy for both humans and machines to read. The researchers also built a system to trace references within these documents. If a specification mentions a specific file or a task that needs to be done, the system checks whether that file actually exists in the project or if the task was ever completed. This revealed a significant gap in some projects: there are thousands of specifications that mention files or tasks that never actually appeared in the code. This suggests that while the practice of writing specifications is growing, the discipline of ensuring those specifications are fully realized is still evolving.

The collection also offers a window into who is doing the writing. Because the data includes the history of every change, researchers can see whether a specification was written by a human, generated by an AI, or a mix of both. They can track how often these documents are updated and how long they remain active before being abandoned. This level of detail allows for questions that were previously impossible to answer, such as whether a well-written specification leads to fewer errors in the final code, or whether certain types of projects are more likely to adopt this new way of working. The data shows that the practice is spreading rapidly, with the vast majority of these files appearing in just the last two years, marking a distinct new era in software development.

The researchers made this entire dataset available to the public, organized in a way that allows anyone to explore the connections between ideas and code. They provided the raw text of the specifications, the history of the projects, and the links between the two, all without hiding the identities of the contributors or the projects, as everything comes from public sources. This transparency means that other scientists can verify the findings, ask new questions, and study how this practice changes over time. The work does not claim that this new method is perfect or that it has solved all the problems of software development. Instead, it provides the first solid foundation of evidence, a large-scale map of a territory that was previously uncharted. It shows that while the tools to write specifications are here and the volume of work is growing, the way humans and machines collaborate to turn those words into software is still being figured out, one project at a time.