Governing AI Coding Agents: An Empirical Analysis of Repository Artifacts
This study empirically analyzes 555 GitHub repository governance artifacts to reveal that while they effectively facilitate knowledge transfer and coding standards, they lack robust oversight mechanisms and show no systematic correlation with organizational maturity or domain, highlighting significant liabilities like vague instructions and fragmentation.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern world of software creation, a new kind of worker has arrived: the AI coding agent. These are not simple tools that just finish a sentence of code for a human; they are autonomous programs capable of reading entire projects, understanding complex instructions, and writing or fixing software across many files on their own. As these digital workers become more common, companies and open-source communities face a critical challenge: how do you manage them? If a human employee makes a mistake, a manager can step in, review the work, and correct the course. But when an AI agent operates with speed and independence, the traditional methods of supervision can feel too slow or too distant. To bridge this gap, developers have started creating special instruction files—digital rulebooks stored directly within the project's code. These files, with names like AGENTS.md or CLAUDE.md, are meant to tell the AI exactly how to behave, what rules to follow, and how to fit into the team's workflow. They are the written constitution for the machine worker.
The question researchers asked was whether these digital rulebooks actually represent a mature system of control, or if they are merely a collection of notes. In the world of organizational management, having a written policy is not the same as having the ability to enforce it. A true governance capability means an organization can not only write down rules but also monitor them, ensure they are followed, and adapt them as conditions change. To find out if the software world has reached this level of maturity with its AI agents, a team of researchers from LUT University in Finland conducted a large-scale investigation. They did not just read a few examples; they gathered and analyzed 555 of these governance files from public software projects on GitHub. Their goal was to see if the presence of these files meant the projects had developed a sophisticated way of managing AI, or if the files were simply documenting expectations without the machinery to back them up.
The researchers approached this by treating the files as evidence of how projects try to govern their AI workers. They first read through a sample of the files to build a detailed map of what was actually inside them. They discovered that these files generally fell into two categories: helpful instructions and hidden weaknesses. The helpful instructions, which they called governance practices, included things like explaining the project's history, defining coding styles, and outlining the steps for a task. The weaknesses, which they termed governance liabilities, were problems that could make the AI's job harder or riskier. These included vague commands that didn't specify how to measure success, instructions that were copied from other projects without being adapted, and security gaps where sensitive information was mentioned without protection.
Once they had this map, the researchers used a powerful language model to scan all 555 files, checking for the presence of these specific practices and weaknesses. They then compared the results against the maturity of the projects. A common assumption might be that older, more popular projects with thousands of stars and many contributors would have better, more robust governance files than smaller, newer ones. The logic would be that as a project grows, it learns from its mistakes and builds better systems. However, the data told a different story. The researchers found no link between the size or age of a project and the quality of its governance files. A massive, well-established project was just as likely to have a weak or vague set of instructions as a small, newer one. Similarly, they looked at whether projects in high-stakes fields, like infrastructure or mobile apps, had stricter rules than those in general web development. Again, there was no difference. The files looked the same regardless of the potential consequences of a mistake.
The most striking finding was about the nature of the governance itself. The researchers found that the files were overwhelmingly focused on instruction rather than oversight. About 82 percent of the files contained instructions on how to transfer knowledge or follow coding standards. In contrast, only about 11 percent of the files included a "human approval gate"—a specific rule requiring a human to check and approve the AI's work before it could be finalized. This suggests that the current approach is largely about telling the AI what to do, rather than building a system to catch it when it goes wrong. Furthermore, the researchers noticed a curious pattern: the more instructions a file contained, the more weaknesses it tended to have. It was not the case that adding more rules cleaned up the mess; instead, adding more rules often added more confusion, redundancy, or contradictions. The files were not becoming more coherent as they grew; they were becoming more cluttered.
The study concludes that while software projects are rapidly adopting these governance files to guide their AI agents, this adoption does not yet signal a mature organizational capability. The files are effective at documenting knowledge and setting expectations, but they lack the structural mechanisms—like accountability, consistent oversight, and project-specific adaptation—that characterize a truly institutionalized system. The presence of a rulebook does not mean the organization has the ability to enforce it. Instead, the current landscape is dominated by "instruction-oriented governance," where the priority is enabling the AI to work, while the systems to control and verify that work remain underdeveloped. For organizations looking to scale their use of AI, the research suggests that simply writing more instructions is not the solution. True governance requires moving beyond documentation to build the organizational routines that can monitor, verify, and adapt to the actions of these autonomous workers.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.