Early Adoption of Agentic Coding Tools by GitHub Projects
This study analyzes over 25,000 agent-generated pull requests across 2,300 GitHub repositories to reveal that while intensive adoption of agentic coding tools remains concentrated in a small subset of projects, successful integration relies heavily on single-human oversight models and varies significantly by project size and productivity.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of software development as a massive, bustling construction site where millions of people are building digital skyscrapers, one line of code at a time. For years, the workers on this site have used "smart assistants"—tools that suggest the next brick or help fix a wobbly wall. But recently, a new kind of worker has arrived: the "agentic" tool. Think of these not as assistants who just whisper suggestions, but as semi-autonomous interns. These digital interns can look at a blueprint, build a whole new room, test it to make sure the lights work, and even knock on the project manager's door to ask, "Can we put this in?" This shift is exciting because it promises to speed up how we build software, but it also raises a big question: How do real human teams actually work with these new digital interns? Do they trust them with the keys to the whole building, or do they keep a tight leash? This is the story of how open-source projects are trying to figure out the rules of this new human-agent partnership.
The paper "Early Adoption of Agentic Coding Tools by GitHub Projects" acts like a detective story, investigating how thousands of software projects are handling these new digital interns. The researchers looked at a massive pile of data: 25,264 "pull requests" (which are basically formal requests to add new code) from 2,361 different projects on GitHub. They focused on a specific three-month window (May, June, and July 2025) and tracked three popular AI coding agents: Copilot, Codex, and Claude Code. Their goal was to answer three main questions: How many projects are actually using these tools? How productive are these projects when using them? And, most importantly, how are humans and agents working together?
Here is what the investigation revealed, and it's a bit more nuanced than you might expect.
First, the researchers found that while these tools are being used in thousands of places, the "heavy lifting" is happening in a very small corner of the construction site. If you picked a random project, the median (the middle point) only generated one to two agent-made requests during those three months. It's like finding that while a new type of power tool exists, most construction crews are only using it once or twice a month. The study suggests that "intensive adoption" is still concentrated in a relatively small subset of projects. However, there is a twist: the size of the team matters a lot. Small projects (with 1 to 5 contributors) were much more eager to jump in. They had a higher percentage of their team members using the tools and, on average, generated way more agent activity than the massive, corporate-sized projects. In fact, small projects averaged about 50 agent requests per repository, while medium and large projects averaged only 5 to 7. It seems the smaller crews are more willing to let the digital interns run wild, while the big teams are still being very cautious.
Second, the team looked at productivity, asking: "How much code is actually getting built per person?" They compared their findings to an industry benchmark that suggests a human developer might handle about 36 requests in three months. The results showed that most projects fell below this line. Only a tiny fraction—just 25 out of the 2,361 projects (about 1%)—managed to exceed that 36-request mark. This suggests that while the tools are capable, most teams aren't yet generating a massive volume of agent work. The few projects that did go above the line were mostly the small ones, indicating that high productivity with these tools might depend on specific project conditions that we don't fully understand yet.
Finally, the study looked at the "dance" between humans and agents. Who is actually in charge? The answer is overwhelmingly one person. In nearly 79% of the cases, a single human developer was the one reviewing the agent's work and also the one merging it into the project. It's a "solo oversight" model. Imagine a teacher grading a student's homework and then immediately putting it in the gradebook without asking a colleague for a second opinion. While larger projects showed a bit more teamwork (with multiple people reviewing), the vast majority of the time, the human-agent collaboration is a one-on-one relationship. The paper suggests that even with these powerful new agents, the human role of reviewing and approving remains central and hasn't been replaced by a team-based safety net.
In short, this paper paints a picture of an early experiment. The digital interns are here, and they are working hard in some small, agile teams, but for most projects, they are still just getting their feet wet. The study suggests that for these tools to truly transform software development, it's not just about making the agents smarter; it's about how human teams organize themselves to trust, review, and manage the work these agents produce. The future of this partnership depends on finding the right balance between letting the agents run and keeping a human hand firmly on the steering wheel.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.