AI Policy, Disclosure, and Human in the Loop: How Are Contribution Guidelines Adapting to GenAI?
This empirical study of 1,000 GitHub repositories reveals that while the majority of open source projects now permit AI-assisted contributions, most enforce strict requirements for disclosing AI usage and maintaining human oversight in the development process.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the world of software development as a massive, bustling city where millions of people are constantly building, fixing, and expanding a giant, shared skyscraper called "Open Source." For years, this city ran on a simple rule: if you want to add a new room or fix a broken window, you have to do the work yourself, and the city's volunteer inspectors (called "maintainers") check your blueprints to make sure they are safe. But recently, a new kind of construction crew arrived: Generative AI. Think of these as super-fast, magical robots that can draft entire blueprints in seconds. While this sounds like a dream, it has created a bit of a traffic jam. The robots are so fast that they are flooding the city with thousands of new blueprints every day, overwhelming the human inspectors. Some of these robot-drafted plans are brilliant, but others are "AI slop"—messy, low-quality drafts that look okay at a glance but fall apart when you try to build with them. The big question for the city is no longer just "Can we build?" but "How do we let the robots help without letting them take over the construction site?"
This paper dives into that exact question by looking at the "rulebooks" (contribution guidelines) of 1,000 of the most popular software projects on GitHub. The researchers wanted to see how these projects are adapting to the robot workforce. They found that most projects aren't banning the robots; instead, they are trying to set up a "Human-in-the-Loop" system. Imagine a construction site where the robot can mix the concrete and lay the bricks, but a human foreman must stand right there, holding the trowel, understanding every step, and signing off on the work before it becomes part of the building. The study analyzed 118 specific rulebooks that mention AI and discovered a clear pattern: while 78% of projects say "Yes, you can use AI," they almost always demand two things. First, you must raise your hand and say, "I used a robot for this" (Disclosure). Second, you must prove that you, the human, actually understand what the robot built (Human in the Loop).
The researchers found that 51% of the projects explicitly require you to disclose when you've used AI, treating it like a safety label on a food package. They want the inspectors to know, "Hey, this part was drafted by an AI, so please double-check it." Another 27% allow AI but don't explicitly cheer for it, while a smaller group (22%) says, "No robots allowed; we want 100% human-made code." However, the most striking finding is that 74% of the projects insist on a human being in the loop. This means that even if a robot writes the code, a human must read it, understand it, and take full responsibility for it. The paper suggests that without this human oversight, the city risks being flooded with "AI slop"—low-quality code that wastes everyone's time.
Interestingly, the paper notes that while many projects have these rules, the instructions can sometimes be a bit fuzzy. Some rulebooks say "disclose if the AI did a significant amount of work," but they don't always define exactly what "significant" means. It's like a teacher saying, "Tell me if you used a calculator," but not saying if that means for one math problem or the whole test. The authors suggest that projects need to be more precise to avoid confusion. They also warn that while some projects are trying to block low-quality AI submissions by closing them automatically, this might accidentally annoy real human contributors who just happen to use AI tools. Ultimately, the paper concludes that the future of open source isn't about choosing between humans and robots, but about making sure the human stays firmly in the driver's seat, using the robot as a powerful tool rather than letting it drive the car.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.