Collaborator or Assistnat? How AI Coding Agents Partition Work Across Pull Request Lifecycles
This paper analyzes 29,585 pull request lifecycles across five AI coding tools to establish a "Collaborator-Assistant" taxonomy that reveals how operational agency is increasingly delegated to agents while merge governance remains predominantly human, highlighting a critical decoupling between execution and oversight in automated software development.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine software development as a busy kitchen where a team of chefs (humans) and a new generation of robotic sous-chefs (AI agents) are working together to create dishes (code updates). The paper you're asking about is a study of how these chefs and robots actually share the work when they submit a new dish to the head chef for approval.
The researchers looked at nearly 30,000 "orders" (called Pull Requests or PRs) from five different AI tools to see who starts the cooking, who does the tasting, and who finally gives the "okay" to serve the dish.
Here is the breakdown of their findings in simple terms:
1. The Two Types of Robotic Chefs
The study found that the AI tools fall into two distinct categories, like two different types of kitchen assistants:
The "Collaborator" Robots (Cursor, Devin, Copilot):
These are like autonomous sous-chefs. You give them a vague idea (like "make a spicy pasta"), and they go into the kitchen, chop the vegetables, cook the sauce, plate the dish, and bring it to the counter for you to taste. They do almost all the heavy lifting. However, they still wait for a human to look at the dish, taste it, and give the final "yes" before it gets served.- The Stat: In these workflows, the robot starts the work 96% of the time.
The "Assistant" Robots (OpenAI, Claude):
These are more like smart recipe generators. You (the human) are still the head chef. You tell the robot exactly what to do, or you start the cooking yourself, and the robot just helps you chop an onion or suggests a spice. The human is driving the car; the robot is just the GPS.- The Stat: In these workflows, the human starts the work 95%+ of the time.
2. The Big Surprise: Who Holds the Keys?
The most interesting finding is about who gets to press the "Serve" button (merge the code).
Even though the "Collaborator" robots do almost all the cooking and bring the dish to the counter, humans still hold the keys to the restaurant 99.9% of the time.
- The robots are allowed to cook and plate the food.
- The robots are rarely allowed to decide if the food is good enough to serve.
- Even when a robot does press the button to serve the dish, the study found that usually, a human had already tasted it and said "good to go" earlier. The robot just clicked the button to finish the paperwork.
The Metaphor: Think of the AI as a Junior Intern. The intern is great at drafting the report, fixing the typos, and printing the document. But the Manager (the human) is the only one who signs the final approval. The intern does the work; the manager takes the credit (and the blame) if something goes wrong.
3. The "Silent Merge" Mystery
The researchers found a tiny number of cases (less than 1%) where a robot seemed to approve its own work without a human tasting it first.
However, the study points out a blind spot in their data. The computer logs show who clicked the button, but not who made the decision.
- Scenario A: A human tasted the food, said "Great," and then set up a rule so the robot could automatically serve it later. (The robot clicked the button, but the human made the decision).
- Scenario B: The robot actually decided to serve the food on its own.
The study says they can't tell the difference between these two scenarios just by looking at the logs. It's like seeing a door open and knowing a robot pushed it, but not knowing if a human told the robot to push it five minutes ago.
4. What This Means for the Kitchen
The study concludes that right now, we are in a "Junior Intern" phase of AI.
- Collaborator tools are great at taking the initiative and doing the work, but they still need a human to double-check everything.
- Assistant tools are great at helping humans who are already working, but they don't take the lead.
The researchers warn that while the robots are getting faster at cooking, the "keys to the restaurant" (the authority to merge code) are still firmly in human hands. The logs show us who did the work, but they don't always show us who was really in charge of the final decision.
In short: AI is doing the heavy lifting, but humans are still the ones holding the clipboard and signing the checks.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.