Technical Summary: Formal Skill: Programmable Runtime Skills for Efficient and Accurate LLM Agents

1. Problem Statement

Large Language Model (LLM) agents are increasingly deployed in real-world workspaces where their ability to act depends on the reliability of their tools and skills. Current approaches to defining agent skills suffer from four primary limitations due to their reliance on Informal Skills (procedural knowledge encoded primarily in natural language):

1. Token Inefficiency: Procedural instructions (e.g., SKILL.md files, Markdown packs) must be loaded as prompt tokens for every relevant turn, leading to high costs.
2. Ambiguity: Natural language is only semi-structured to a machine, making it difficult to enforce strict ordering, safety rules, or completion criteria.
3. Weak Enforcement: Constraints such as "verify before finishing" or "do not edit protected files" are often soft suggestions in prompts rather than executable checks, allowing models to bypass them.
4. Brittle Recovery: State and recovery context are implicit in the conversation transcript rather than being represented as explicit, skill-local runtime state, leading to ad-hoc recovery strategies in long-horizon tasks.

Existing systems, such as those using SKILL.md libraries or schema-based tool interfaces (e.g., MCP, function calling), either treat procedures as documents to be read or define individual actions without governing the workflow state, policy enforcement, or completion discipline.

2. Methodology: Formal Skill and FairyClaw

The authors propose Formal Skill, a runtime-native abstraction that converts reusable agent capabilities from natural-language artifacts into structured, executable objects. This approach shifts the burden of procedural invariants from the model's prompt to the runtime's execution logic.

Core Components of Formal Skill

A Formal Skill consists of five integrated components:

1. JSON Metadata and Action Schemas: Defines the machine-readable interface, replacing long procedural text with compact tool descriptions and argument schemas.
2. Reliable Python Executors: Implements actions with deterministic validation and bounded side effects, ensuring that tool calls are safe and correct before execution.
3. Hook-Governed Control Logic: A pipeline of hooks (e.g., before_llm_call, before_tool_call, after_tool_call) that mediate model and tool interactions. These hooks enforce policies, filter visible tools based on state, and validate arguments.
4. Skill-Local Runtime State: An explicit state object (e.g., current phase, verification status, artifact gates) that tracks progress. This state is consumed by hooks to determine tool visibility and task completion, replacing implicit transcript-based recovery.
5. Routing Metadata: Determines when and how a skill is exposed to a specific task or subtask, allowing for dynamic capability selection.

Implementation: FairyClaw

The authors implement Formal Skill in FairyClaw, an open-source, event-driven agent runtime. Key architectural features include:

• Single-Step Planner: The runtime executes one LLM decision and its tool effects at a time, persisting the turn and updating skill state before scheduling the next event.
• Typed History IR: Maintains a typed intermediate representation of history.
• Sub-Agent Delegation & Routing: The parent planner can create sub-sessions with narrowed workspaces and routed skill sets. This ensures the model only sees tools and guidance relevant to the current phase, reducing token usage and action ambiguity.
• Hook Pipeline: A structured pipeline where state is converted into tool visibility, prompt guidance, and policy checks.

Case Study: CODEREPAIROPS

To instantiate the concept, the authors developed CODEREPAIROPS, a formal skill for code repair and test-driven debugging.

• Phases: The skill enforces a workflow of reproduce → diagnose (optional) → patch → verify → report.
• State-Driven Visibility: In the reproduce phase, only evidence collection tools are visible. In patch, only unified patch tools are allowed, with guards rejecting protected paths or oversized changes.
• Gate Enforcement: Completion is blocked until verification passes and required artifacts are produced. If gates fail, the session is forced to return to the previous phase rather than allowing premature completion.

3. Key Contributions

1. Formal Skill Abstraction: A new paradigm for agent skills that moves procedural knowledge from natural-language prompts into JSON metadata, action schemas, reliable executors, hook-governed control loops, and skill-local runtime state.
2. FairyClaw Runtime: An open-source event-driven runtime that makes Formal Skills routable, executable, observable, and enforceable. It integrates single-step planning, persistent session state, and hook-based execution control.
3. Empirical Validation: Demonstration on the Harness-Bench suite showing that Formal Skills can achieve competitive performance while significantly reducing token consumption.

4. Experimental Results

The authors evaluated FairyClaw against five other agent harnesses (Moltis, NullClaw, ZeroClaw, Hermes, OpenClaw) using two model families (gpt-5.4 and gemini-3.1-pro) across 75 tasks.

• Overall Performance:
  • On gpt-5.4, FairyClaw achieved the highest combined score (0.746) among all agents, ranking first in this group and third overall across both model families.
  • On gemini-3.1-pro, FairyClaw achieved a competitive score of 0.635, ranking third overall.
• Token Efficiency:
  • FairyClaw used substantially fewer tokens than all comparators. On gpt-5.4, it used 46.8K tokens per task on average, which is approximately 48% lower than the mean of the other five harnesses and 33% lower than the next most efficient system.
  • Total token usage for FairyClaw (gpt-5.4) was 3.51M, compared to 6.29M for Moltis and 7.28M for Hermes.
  • The authors note that context compression was disabled during these runs; thus, the efficiency gains are attributed to Formal Skill routing, compact phase guidance, and narrower tool exposure rather than transcript summarization.
• Task-Specific Performance:
  • On a representative code-debugging task requiring evidence collection, workspace modification, and verification, FairyClaw achieved the highest combined scores for both models (0.865 for gpt-5.4 and 0.843 for gemini-3.1-pro).
  • While FairyClaw sometimes required more rounds (e.g., 48 rounds vs. 21 for NullClaw on gpt-5.4), these rounds were governed by explicit state transitions and hook controls rather than unstructured prompting, resulting in lower overall token consumption despite the longer trajectory.

5. Significance and Claims

The paper claims that Formal Skill represents a shift in how reusable agent capabilities are constructed. By treating skills as "runtime contracts" rather than "textual manuals," the approach:

• Decouples Reasoning from Execution: The model retains the role of reasoning and choosing among structured actions, while the runtime owns the procedural invariants, safety rules, and state management.
• Enforces Discipline: Constraints become executable checks (e.g., blocking unsafe edits or premature completion) rather than instructions the model may choose to ignore.
• Improves Efficiency: It drastically reduces token consumption by replacing long procedural documents with compact, state-conditioned guidance.

The authors position this not as a replacement for natural language in agent behavior, but as a reassignment of roles: natural language is used for task-specific reasoning, while reusable procedures are encoded as compact, enforceable runtime objects. The results suggest that this approach allows agents to perform complex, multi-step tasks with higher reliability and significantly lower cost.