Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
This empirical study of 9,427 agentic pull requests reveals distinct usage patterns between core and peripheral developers, showing that while peripheral developers delegate a broader range of tasks and are more likely to merge without verification, core developers focus on documentation and testing while maintaining stricter integration standards.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a software project as a massive, bustling construction site. For years, the workers (developers) have used smart assistants (like chatbots) to help them write blueprints. But recently, a new type of worker has arrived: the Autonomous Agent. These aren't just assistants who wait for instructions; they are like independent robots that can grab a hammer, fix a leak, write a manual, and even submit their own work for approval without being told exactly how to do every step.
The researchers in this paper wanted to know: Do the "Bosses" (Core Developers) and the "New Hires" (Peripheral Developers) use these robots in the same way?
To find out, they looked at nearly 10,000 work submissions (called Pull Requests) from GitHub, where these robots had done the heavy lifting. Here is what they discovered, broken down into simple stories:
1. Who is using the robots, and for what?
- The New Hires (Peripheral Developers): A small group of new or less-experienced workers are using these robots a lot. They treat the robots like a "Swiss Army Knife," handing them all kinds of tasks equally: fixing broken windows (bugs), building new rooms (features), writing instruction manuals (documentation), and testing the plumbing (testing).
- The Bosses (Core Developers): The experienced leaders use the robots too, but they are more picky. They mostly send the robots to do the boring, repetitive chores: writing manuals and running tests. They rarely ask the robots to fix complex bugs or build new features.
- The Result: Even though both groups use the robots, the Bosses' robot-made work gets approved and merged into the main building much more often than the New Hires' work. It seems that having a "reputation" still matters, even when a robot did the work.
2. How do they inspect the robot's work?
Imagine the robots submit their finished work to a quality control inspector.
- The Bosses: They are the stricter inspectors. They write slightly more comments and questions. When they find a problem, they don't just say, "This looks messy." They often say, "Here is a completely different way to build this that would be stronger." They focus on the long-term health of the building.
- The New Hires: They also inspect the work, but they focus more on the immediate look and feel. They might say, "Hey, these bricks are stacked in a weird order; let's rearrange them." They care about organization and structure.
- The Surprise: Both groups mostly worry about whether the building can be easily expanded or changed in the future (evolvability).
3. Do they fix the robot's mistakes?
- Mostly No: In about 74% of cases, the work submitted by the robot was good enough that the humans didn't touch it at all. They just hit "Approve."
- When they do fix it: If a human does step in to change the robot's code, they usually do a "refactor." Think of this as tidying up a messy room—moving furniture around to make it flow better, rather than rebuilding the walls.
- New Hires tend to use the robot, then step in to fix specific broken things (bugs).
- Bosses tend to step in to improve the documentation (the instruction manuals) and clean up the structure.
4. The "Safety Check" (CI Pipeline)
Before a new room is added to a building, safety inspectors run a series of automated tests (like checking if the fire alarms work).
- The Bosses: They are very strict. They almost never let a robot's work into the main building unless all the safety tests pass. They act as the ultimate gatekeepers.
- The New Hires: They are more willing to let the robot's work in even if some safety tests are still running or failed. About twice as often as the bosses, they merge the work without waiting for the full safety report.
The Big Takeaway
The paper concludes that experience still matters, even when robots are doing the work.
- The "Toil" Factor: Both groups use robots to get rid of the boring, tedious work (like writing manuals and running tests), which frees them up for more important thinking.
- The Trust Gap: New hires are trusting the robots a bit too quickly, sometimes merging work before it's fully checked. The experienced leaders are still holding the reins, ensuring that the final product is safe and high-quality.
- The Lesson: We shouldn't assume that because a robot did the work, everyone will treat it the same. The human experience level changes how the robot is used, what it is asked to do, and how carefully its work is checked.
In short, the robots are great tools, but the "Bosses" are still the ones making sure the building doesn't fall down.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.