💻 computer science

AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates

This paper empirically analyzes the AIDev-pop dataset to reveal that concurrent AI agent submissions are highly prevalent, predominantly involve the same agent, and result in significantly higher merge conflict rates for cross-agent pairs compared to intra-agent ones.

Original authors: George Xu, Arjun Subramanian, Nithilan Karthik

Published 2026-07-08

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: George Xu, Arjun Subramanian, Nithilan Karthik

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, bustling construction site where instead of human workers, you have dozens of highly skilled, independent robots (AI coding agents) all trying to build and fix the same house at the exact same time.

This paper is a report card on what happens when these robots try to work together without talking to each other. The researchers looked at thousands of real-world projects on GitHub to see how often these robots clash, what their fights look like, and how messy the results get.

Here is the breakdown of their findings using simple analogies:

1. The "Traffic Jam" of Robots

The first big question was: How often are these robots working on the same house at the same time?

The answer is: Almost always.

The Finding: In about 40% of the projects, two robots were actively working on the house at the exact same second. If you give them a little more time (a one-week window), that number jumps to nearly 95%.
The Analogy: Imagine a busy kitchen where 9 out of 10 chefs are chopping vegetables on the same cutting board at the same time. It's chaotic, but it's the norm.
Who is fighting? Surprisingly, it's rarely a fight between different types of robots (like a "Devin" robot fighting a "Copilot" robot). It's almost always the same type of robot fighting itself. In 99.5% of the cases, the conflict is between two instances of the same AI model working in parallel. It's like having two identical twins trying to paint the same wall without coordinating.

2. The "Clash" (Merge Conflicts)

When two robots try to save their changes to the same file, the computer has to decide which version to keep. This is called a "merge." If they changed the same lines, the computer gets confused and throws up a "Conflict" flag.

The researchers simulated these merges to see how often things broke:

Same Robot vs. Same Robot: When two instances of the same AI tried to merge, they clashed about 20% of the time.
Different Robots vs. Different Robots: When two different types of AI tried to merge, they clashed about 42% of the time.
The Takeaway: Different robots are twice as likely to disagree with each other than identical robots are. However, since identical robots are the ones working together most often, the overall mess is mostly caused by them.

3. What Are They Fighting About?

When the robots do clash, what is the nature of the fight?

The Location: Most fights happen in the source code (the actual instructions of the software), not in the "shopping lists" (dependency files) or documentation. About 84% of the conflicts were in the code itself.
The Type of Fight:
- Content Fights (58%): Both robots tried to change the same sentence in the same paragraph.
- Structural Fights (42%): This is the weirdest part. One robot decided to delete a file, while the other decided to modify it. Or, both robots tried to create a brand new file with the same name but different contents.
- The Analogy: It's like one robot saying, "I'm tearing down the kitchen," while the other says, "I'm renovating the kitchen," and a third says, "I'm building a new kitchen right here." They can't all be right.

4. Why Does This Matter? (The Hidden Costs)

The paper notes that while we only measured the "textual" fights (the code lines), the real cost is much higher.

The "Lost Time" Cost: Because the robots don't know the others are working, they waste time building features that can never be combined.
The "Human Cleanup" Cost: Eventually, a human has to step in and fix the mess. They have to decide which robot was right, which file to keep, and how to un-delete things. This defeats the purpose of having robots do the work in the first place.

Summary

The paper concludes that right now, AI coding agents are like lonely geniuses working in a shared room. They are incredibly productive individually, but because they don't talk to each other (even when they are the same model), they constantly trip over their own feet.

Frequency: They are almost always working concurrently.
Conflict Rate: They clash about 20% of the time (same robot) and 42% of the time (different robots).
The Mess: The fights are mostly about code structure (deleting vs. adding files) rather than just simple typos.

The researchers suggest that for AI to truly help build software, we need to teach these robots how to coordinate their schedules and talk to each other before they start hammering on the same code.

Technical Summary: AI Agent Pull Requests on GitHub

Problem Statement

While AI coding agents have evolved to operate independently, assessing issues, writing fixes, and submitting Pull Requests (PRs) autonomously, a critical gap exists in understanding the implications of concurrent agent activity. Previous research has focused on the quality of individual agent outputs or conflicts between a single agent's PR and its base branch. However, there is no empirical examination of what happens when multiple agents (or multiple instances of the same agent) work on the same repository simultaneously. Specifically, the literature lacks data on:

The frequency of concurrent agentic workflows on GitHub.
The likelihood that these simultaneous workflows will conflict with one another.
The structural nature of these conflicts (e.g., text vs. semantic).

This paper addresses the "coordination gap" between architecture and text in autonomous development, noting that while version control tools can detect line-level text conflicts, they cannot inherently resolve conflicting intents among multiple autonomous developers.

Methodology

The study utilizes the AIDev-pop dataset, comprising 33,596 PRs across 2,807 repositories. The authors employ a four-part empirical approach:

Co-activity Formalization: The authors define "co-active" PRs based on temporal overlap. A PR $P_i$ is defined by its repository, authoring agent, open time ( $T_{open}$ ), and close time ( $T_{close}$ ). Two PRs are considered co-active if their activity windows $[T_{open} - k, T_{close} + k]$ overlap. The study analyzes this across four time-padding thresholds ( $k \in \{0, 1, 3, 7\}$ days).
Stratified Sampling: To avoid skewing results from highly active repositories, the authors created a stratified sample for merge replay:
- Stratum A (Intra-agent): 625 repositories, each contributing one pair of PRs from the same agent.
- Stratum B (Cross-agent): All 122 repositories exhibiting cross-agent activity, each contributing one pair of PRs from different agents.
- Total Sample: 747 unique co-active pairs.
Large-Scale Merge Replay: The authors developed an automated, multi-threaded process using headless git merge-tree functionality. This simulates a three-way merge in memory without touching the working directory. The process fetches the head commits of both PRs, identifies the common ancestor, and attempts a merge.
- Conflict Detection: A return code of 1 indicates a textual conflict. The system extracts conflicting file paths and conflict types (e.g., content, modify/delete, add/add).
- Scope: The study focuses strictly on textual conflicts (Layer 1). The authors note this provides a "conservative lower bound" for total system friction, as build (Layer 2) and semantic (Layer 3) conflicts cannot occur without a successful textual merge.
Taxonomy Analysis: Conflicts are categorized by file type (source code, dependencies, config, etc.) and structural type (content overlap vs. structural changes like file deletion).

Key Contributions and Results

1. Frequency of Concurrency (RQ1)

Concurrent agent activity is pervasive.

Strict Overlap ( $k=0$ ): 40.2% of repositories contain at least one pair of co-active PRs.
PR-Level Prevalence: 79.4% of all agent-authored PRs are open concurrently with another agent PR.
Expanded Windows: When allowing a ±7 day window, 53.4% of repositories and 95.0% of PRs exhibit co-activity.

2. Composition of Co-activity (RQ2)

The vast majority of concurrency is intra-agent, not cross-agent.

Intra-agent: 99.5% of co-active pairs are authored by the same agent model.
Cross-agent: Only 0.5% of pairs involve different agents, occurring in just 122 of the 2,807 repositories (approx. 4.3%).
Dominance: OpenAI Codex is the primary driver of concurrency, acting as the "majority anchor" for cross-agent interactions due to its high volume.

3. Merge Conflict Rates (RQ3)

There is significant friction when merging concurrent agent PRs, with a clear distinction between intra- and cross-agent scenarios:

Intra-agent Conflict Rate: 19.8% (119/601 pairs).
Cross-agent Conflict Rate: 41.7% (48/115 pairs).
Statistical Significance: The 95% confidence intervals for these rates are non-overlapping, indicating a clear distinction in conflict likelihood between the two categories.

4. Conflict Structure and Taxonomy

The nature of the conflicts reveals deep coordination failures:

File Types: 84.4% of conflicted files are source code, contradicting the assumption that dependency manifests are the primary source of conflict.
Structural vs. Content:
- 57.6% are content conflicts (different updates to the same lines).
- 41.9% are structural conflicts (26.8% modify/delete and 15.1% add/add).
Implication: Agents frequently disagree on whether a file should exist at all or independently create versions of the same new file, indicating a lack of awareness of parallel operations.

5. Associations (RQ4)

Throughput: PR-level co-activity increases with repository throughput (from ~53% in low-throughput repos to ~90% in high-throughput repos).
Agents: Devin (82.5%) and OpenAI Codex (80.9%) show the highest PR-level co-activity rates.
Languages: Go (92%) and Ruby (87%) repositories show the highest co-activity rates, while C++ (56%) and HTML (44%) show the lowest.

Significance and Claims

The paper claims to provide the first empirical examination of concurrent agent submissions. Its primary significance lies in challenging the theoretical assumption that multi-agent orchestration is currently a problem of heterogeneous vendor coordination. Instead, the data reveals that the dominant operational issue is the management of high-frequency output from a single agent framework.

The authors assert that:

Lack of Awareness: Current agents operate in isolation without "horizontal awareness" of other agents (even of the same type) modifying the same files.
Conservative Estimates: The reported conflict rates (19.8% and 41.7%) are conservative lower bounds. They represent only textual conflicts; the actual cost includes unmeasured build failures, semantic contradictions, CI/CD compute waste, and token budget depletion.
Structural Disagreement: The high rate of structural conflicts (modify/delete, add/add) suggests that agents are not merely editing lines but are fundamentally disagreeing on the repository's structure, a problem that simple code review cannot easily resolve.

The study concludes that uncoordinated reactive development paradigms are insufficient for modern autonomous engineering and that future research must focus on programmatic orchestration and live workspace communication protocols to mitigate these integration frictions.