Authoring Agent Skills: A Software-Engineering Approach
This paper advocates for applying software-engineering principles to the authoring of Agent Skills—reusable procedural knowledge for large language model agents—by defining their structure, distinguishing them from other behavioral mechanisms, and establishing an evaluation-driven process for their creation and use.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you've just invited a super-smart, incredibly fast robot assistant into your room to help you build a treehouse. This robot knows everything about wood, nails, and physics, but it doesn't know your specific style. It doesn't know that you always paint the railing blue, or that you have a weird rule about never using nails shorter than two inches. If you want the robot to build the treehouse exactly how you like it, you have to tell it. But if you have to explain every single tiny detail every time you ask for help, you'll get exhausted, and the robot might get confused.
This is the world of "AI Agents"—smart computer programs that can do tasks for us. To make them truly helpful, developers are giving them "Skills." Think of a Skill not as a magic spell, but as a little instruction manual or a toolkit you hand the robot. It's a file that says, "Hey, when you see a job like this, here is exactly how I want you to do it." The big question scientists and engineers are asking is: How do we write these manuals so the robot actually follows them? If we write them poorly, the robot might ignore them or get the job wrong. If we write them well, the robot becomes a true expert at your specific way of doing things.
The paper you're about to read, written by Giuseppe Destefanis, argues that writing these "Skills" shouldn't be treated like writing a casual note or a poem. Instead, it should be treated like software engineering. Just like a programmer builds a reliable app with clear rules, clean code, and safety checks, we should build AI Skills with the same care. The paper suggests that a Skill is actually a piece of software in disguise. It has an "interface" (a label telling you what it does) and an "implementation" (the actual instructions). The author proposes that we should use the same strict rules we use for building computer programs—like keeping things simple, separating the "what" from the "how," and testing them carefully—to make sure our AI agents don't go off the rails.
The "Skill" as a Digital Swiss Army Knife
Imagine you have a giant, magical backpack that can hold anything. Inside, you have a special pocket for "Making Release Notes." This pocket isn't just a piece of paper; it's a whole kit. It has a label on the outside that says, "Use me when you need to write a summary of changes for a new version of a game." That label is the interface. The robot looks at the label, sees a task that matches, and decides to open the pocket.
Inside the pocket, you don't just dump a messy pile of notes. You have a clear, step-by-step recipe (the body) and maybe a few extra tools, like a script that automatically counts the changes or a style guide for how the notes should look. This is the implementation. The paper calls this a "Skill," and it insists that this whole kit is a software artifact. That's a fancy way of saying it's a piece of technology that needs to be built, tested, and maintained just like a video game or a website.
The author argues that if you treat a Skill like a casual note, it will fail. If you treat it like a piece of software, it will work. Here is how the paper breaks down the rules for building these digital kits:
1. The "Single Responsibility" Rule
Think of a Swiss Army knife. If you try to make one tool that is a screwdriver, a can opener, a saw, a toothbrush, and a pizza cutter all at once, it's going to be terrible at everything. The paper says a Skill should do one thing really well. If a Skill tries to do "everything related to coding," the robot might get confused about when to use it. But if the Skill is specifically "Drafting release notes from pull requests," the robot knows exactly when to grab it. The paper suggests that a focused Skill is selected more reliably by the AI.
2. The "Staged Loading" Trick
Imagine you have a library with a million books. If you tried to read all of them at once to find the one you need, your brain would explode. The paper explains that AI Skills use a clever trick called staged loading.
- Level 1 (The Label): First, the robot only sees the label on the pocket (the name and a short description). This costs very little "brain power" (tokens).
- Level 2 (The Recipe): If the robot thinks the task matches the label, it opens the pocket and reads the main recipe.
- Level 3 (The Tools): It only pulls out the extra tools (scripts or reference files) if the recipe specifically says, "Oh, I need to check this specific file now."
This means you can have a Skill with a huge library of reference material inside it, and the robot won't get overwhelmed unless it actually needs to use that material. It's like having a massive toolbox but only opening the drawer you need.
3. The "Description" is the Trigger
The most important part of a Skill is the description on the label. The paper warns that if you write a vague label like "Handles releases," the robot won't know when to use it. It needs to be specific: "Drafts release notes from the pull requests merged between two version tags. Use when cutting a release." The robot matches the task to this description. If the description is fuzzy, the robot might miss the Skill entirely, or pick the wrong one. The paper suggests that the description is the "contract" between you and the robot.
4. Testing with "Behavioral Evaluation"
In normal computer programming, you can test a function by running it and checking if the answer is exactly "5." But AI is different. The robot might give you a slightly different answer every time, even if it's doing the right thing. The paper says you can't test a Skill with a simple "pass/fail" check. Instead, you have to use behavioral evaluation. This means you give the Skill a bunch of real-world tasks, run it many times, and see if it gets the job done correctly most of the time. It's like testing a new recipe by cooking it ten times and tasting it, rather than just checking if the ingredients are on the list.
The Great Confusion: Skills vs. Other Tools
One of the paper's biggest contributions is sorting out the confusion between different ways to tell an AI what to do. The author compares Skills to other tools like Hooks, Slash Commands, and Memory Files.
Imagine you are directing a play.
- Memory Files are like a script that is always present on the stage, visible to the actors (the AI) at every single moment of the performance. They provide standing project context that never fades, ensuring the robot always has this information available, even though it doesn't force a specific action.
- Slash Commands are like a stagehand shouting, "Hey, do the magic trick now!" You (the user) have to say the command. The robot waits for you.
- Hooks are like a strict, automated safety mechanism. If the actor tries to jump off the stage without a harness, the safety mechanism deterministically executes a rule to catch them or stop the action. The robot cannot ignore this; it happens every single time the specific event occurs, regardless of what the robot decides.
- Skills are like a specialized procedural toolkit loaded on demand. When the robot encounters a task, it matches the description and loads the specific instructions to handle that domain. It is a source of expert knowledge that the robot actively retrieves to solve a problem, rather than just a passive suggestion.
The paper draws a clear line: If you need something to happen every single time (like a safety check before saving a file), you must use a Hook. If you want the robot to use its brain to decide if a specific procedure is needed, you use a Skill. The paper warns that mixing these up is a common mistake. If you write a safety rule inside a Skill, the robot might decide to skip it. If you write a flexible suggestion inside a Hook, the robot might get stuck doing the same thing even when it shouldn't.
The "Trust" Problem
Finally, the paper touches on a scary but important idea: Trust. If you download a Skill from someone else (a third party), you are letting them write instructions that your robot will follow. That Skill could contain scripts that delete files or send data to the internet. The paper argues that you should treat a Skill exactly like a piece of software you download from the internet. You shouldn't just click "install." You need to read the code, check the scripts, and make sure it's safe. The robot will do whatever the Skill tells it to do, so you have to be the gatekeeper.
The Bottom Line
Giuseppe Destefanis's paper is a call to action for anyone building AI tools. It says: "Stop treating AI instructions like casual notes. Start treating them like serious software."
By using the principles of software engineering—keeping things simple, separating the label from the content, testing with real scenarios, and knowing exactly when to use a Skill versus a Hook—we can build AI agents that are not just smart, but also reliable and safe. The paper doesn't promise that this will solve all AI problems, but it suggests that if we build our Skills with the same care we build our apps, we can make our digital assistants much more trustworthy partners. It's about moving from "hoping the robot gets it right" to "engineering the robot to get it right."
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.