Context Engineering for AI Agents in Open-Source Software
This paper presents a preliminary study of 466 open-source projects to investigate the adoption, content structure, and evolution of AGENTS.md files, revealing significant variation in how developers provide context for AI agents and highlighting the potential for future research on optimizing context engineering to improve generated code quality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are hiring a brilliant, super-fast new intern to help you build a complex machine. This intern is an AI agent. If you just say, "Fix the engine," they might guess wrong because they don't know your specific rules, your preferred tools, or how your team usually works.
This paper is about a new way of giving that intern a "user manual" specifically written for a robot, not a human.
Here is the breakdown of the research in simple terms:
The Problem: The Intern Needs a Handbook
In the past, software developers used tools like GitHub Copilot to help write code. Now, we have "AI Agents" that can do more work on their own, like fixing bugs or building entire features. But just like a human employee, an AI needs to understand the company culture and rules to do a good job.
Vendors (the companies making the AI) suggested that developers create a special file in their project folder. Think of this file as a digital "Employee Handbook" for the AI. It tells the AI things like:
- "We always name our variables this way."
- "Never touch the main file directly."
- "Here is how we test our code."
Recently, a standard format called AGENTS.md was created so all these AI tools could read the same type of handbook.
The Study: Checking the Handbooks
The researchers wanted to see if real-world software projects were actually using these handbooks and, if so, what they looked like. They went on a digital scavenger hunt through 10,000 open-source software projects (projects where the code is free for anyone to see and use).
1. How many people are using them? (Adoption)
They found that only 466 projects (about 5%) had created one of these AI handbooks.
- Analogy: It's like walking into a giant office building with 10,000 rooms and finding that only a few rooms have put up a "New Employee Rules" sign. We are still in the very early days of this trend.
2. What is inside the handbooks? (Content & Style)
The researchers looked closely at the 155 AGENTS.md files they found. They discovered two main things:
No Standard Recipe: There is no single "correct" way to write these files. Some are short, some are long. Some list rules, others just describe how things work.
Different "Voices": The way developers talk to the AI varies wildly. The researchers categorized the writing styles into five types:
- Descriptive: "This is how we do things." (Just stating facts)
- Prescriptive: "Do it this way." (Giving direct orders)
- Prohibitive: "Do NOT do this." (Setting boundaries)
- Explanatory: "Do this because..." (Giving a reason)
- Conditional: "If X happens, then do Y." (Giving situational logic)
Analogy: Imagine 155 different managers trying to write a rulebook for the same intern. One manager writes a strict list of "Don'ts," another writes a friendly story about how the team works, and a third writes a complex "If/Then" flowchart. There is no consistency yet.
3. Do the handbooks change over time? (Evolution)
The researchers tracked how these files changed over time by looking at the "commit history" (the version control log).
- Stability: About half of the files were never changed after they were first created.
- Tweaking: For the files that were changed, the most common updates were adding new instructions or tweaking existing ones.
- Analogy: Once a manager writes the rulebook, they often leave it alone. But when they do update it, they are usually just adding a new rule or clarifying an old one, rather than rewriting the whole thing.
The Big Takeaway
The paper concludes that we are currently in a "wild west" phase of Context Engineering. Developers are experimenting with how to talk to AI agents, but they haven't settled on a standard way to do it yet.
The researchers see these files as a unique "natural laboratory." Because these files are public and version-controlled, we can watch in real-time how humans learn to "speak" to AI agents. The goal isn't to tell developers what to do right now, but to understand how they are currently figuring it out so we can eventually help them do it better.
In short: Developers are starting to write rulebooks for their AI helpers, but right now, every rulebook looks different, and no one is sure which style works best yet.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.