OSS-CRS: Liberating AIxCC Cyber Reasoning Systems for Real-World Open-Source Security

This paper introduces OSS-CRS, an open-source, locally deployable framework that liberates DARPA's AIxCC cyber reasoning systems from obsolete competition infrastructure, enabling their practical application to discover and patch vulnerabilities in real-world open-source projects, as demonstrated by the successful porting of the first-place Atlantis system to find 10 new bugs.

The Gist

Imagine a high-stakes cooking competition where seven teams of brilliant chefs were given a challenge: not just to find a rotten ingredient in a giant, complex kitchen, but to fix it, cook a new dish, and prove it's safe to eat, all without human help.

They succeeded! These "Cyber Reasoning Systems" (CRSs) found bugs in software and wrote the code to patch them. But here's the catch: when the competition ended, the chefs packed up their kitchens. They left behind their recipes, but the kitchens themselves were built on a special, temporary cloud infrastructure that no longer exists. If you tried to use their recipes today, they wouldn't work because they were tied to a specific stove, a specific set of ingredients, and a specific cloud that was turned off.

Enter OSS-CRS: The Universal Kitchen.

This paper introduces OSS-CRS, a new, open-source framework that acts like a universal adapter and a shared kitchen. It takes those competition-winning "recipes" (the AI systems) and lets anyone run them in their own local kitchen to fix real-world software problems.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Locked-in" Chefs

The authors looked at the seven winning teams from the competition and found three big reasons why their systems were useless to the rest of the world:

• Duplication: Every team built their own version of the same basic tools (like their own stoves, sinks, and ingredient trackers) from scratch. It was like seven teams each building their own refrigerator instead of sharing one.
• Cloud Lock-in: The systems were designed to run on a specific, temporary cloud server (like Azure) that was shut down after the contest. Trying to run them now is like trying to drive a car that only works on a track that has been demolished.
• Monolithic Design: The systems were built as giant, single blocks. You couldn't take Team A's amazing "bug-finding robot" and Team B's "patch-writing robot" and combine them. They were stuck together like a solid brick; you couldn't swap out the parts.

2. The Solution: The "Universal Adapter" (OSS-CRS)

The authors built OSS-CRS, which acts as a universal power strip and a shared workspace.

• The Interface (The Power Strip): Instead of needing a specific plug for every system, OSS-CRS gives every AI system a standard plug. Now, any system can plug in and start working immediately.
• The Budget Manager (The Wallet): Using AI (Large Language Models) to fix bugs costs money (like buying ingredients). The competition gave teams a $50,000 budget. OSS-CRS has a built-in "wallet" that tracks how much money each AI spends. If an AI tries to spend too much, the system cuts it off, ensuring you don't go bankrupt while testing.
• The Exchange Table (The Potluck): This is the coolest part. Imagine a potluck dinner.
  • Team A's Fuzzer (a robot that tries to break the software) finds a broken plate.
  • Instead of keeping it, it puts the broken plate on a central table.
  • Team B's Patcher (a robot that fixes things) sees the broken plate on the table, picks it up, and fixes it.
  • They don't need to talk to each other directly; they just share items on the table. This allows different teams' technologies to work together seamlessly.

3. The Result: Real-World Success

To prove it works, the authors took the first-place winner from the competition (a system called ATLANTIS) and "ported" it to this new universal kitchen.

• The Setup: They ran it on a single computer (no giant cloud needed).
• The Test: They pointed it at 8 real-world open-source projects (like libraries used by millions of people).
• The Harvest: In just 24 hours, the system found 10 brand-new bugs (including 3 that were very dangerous) that no one knew about before. It also wrote the code to fix them.

Why This Matters

Before this, if you wanted to use these super-smart AI bug-finders, you had to be part of the specific team that built them or have access to a massive, expensive cloud setup.

OSS-CRS changes the game by:

1. Democratizing Security: Anyone can now run these advanced AI systems to protect their software.
2. Encouraging Collaboration: Researchers can now mix and match the best parts of different systems (like taking the best fuzzer from one team and the best patcher from another) to build even stronger defenses.
3. Saving Money: It manages costs so you don't accidentally spend a fortune on AI credits while testing.

In short: The authors took the "secret sauce" from a high-tech cooking competition, built a universal kitchen where anyone can cook with it, and proved that it can find and fix real-world problems today. It turns a closed, expensive competition into an open, shared tool for keeping the internet safe.

Technical Summary

1. Problem Statement

The DARPA AI Cyber Challenge (AIxCC) successfully demonstrated that autonomous Cyber Reasoning Systems (CRSs) could not only discover software vulnerabilities but also validate them and synthesize patches. Seven finalist teams open-sourced their systems after the competition. However, these systems remain largely unusable in real-world scenarios due to three critical deployment barriers:

1. Infrastructure Duplication: Each team independently rebuilt common platform services (e.g., container orchestration, LLM gateways, artifact storage), leading to redundant engineering efforts.
2. Cloud Lock-in: All systems were tightly coupled to the specific Azure and Kubernetes infrastructure provided during the competition. Since this environment was decommissioned, the systems cannot run locally or on other clouds without massive re-engineering. For example, the first-place system (ATLANTIS) required over 20 Azure VMs and specific SDK calls to scale.
3. Monolithic Design: The CRSs were built as monolithic black boxes. Researchers could not isolate, compare, or combine specific techniques (e.g., a fuzzer from Team A and a patcher from Team B) because no modular interfaces existed.

Consequently, the community cannot run comparable experiments, developers lack a stable integration contract, and security practitioners cannot deploy budget-aware, actionable workflows.

2. Methodology: OSS-CRS Framework

The authors propose OSS-CRS, an open-source, locally deployable framework designed to decouple CRS logic from competition-specific infrastructure. It acts as a platform for developing, running, and composing CRSs against real-world open-source projects (specifically those compatible with OSS-Fuzz).

Key Architectural Components

• Unified Three-Phase Execution Model:

  • Prepare: Builds CRS container images (analysis tools) independent of the target project.
  • Build-Target: Compiles the target project with necessary instrumentation (sanitizers) and creates build snapshots.
  • Run: Launches CRS containers to execute analysis, exchange artifacts, and validate patches.
  • Benefit: Separates setup from execution, enabling caching and modularity.

• Standard Interface (libCRS):

  • A Python library injected into every CRS container.
  • Provides APIs for submitting build outputs, registering shared directories, exchanging artifacts (PoVs, patches, seeds), and validating patches (apply, rebuild, test).
  • Eliminates the need for CRS developers to write custom infrastructure code.

• Resource Management & Isolation:

  • Compute/Memory: Uses Docker cgroups to enforce CPU pinning and memory limits per CRS.
  • Network: Assigns isolated Docker networks to each CRS to prevent interference; inter-CRS communication occurs only via a shared filesystem.
  • LLM Budgeting: Treats LLM API costs as a first-class resource. A LiteLLM proxy routes requests and enforces per-CRS dollar budgets (e.g., $50/campaign), preventing cost overruns.

• Artifact Exchange Mechanism:

  • Uses a file-based "Exchange Sidecar" to synchronize artifacts between CRSs.
  • CRSs write to private submit directories and read from a shared fetch directory.
  • Includes content-hash-based deduplication to prevent redundant processing of the same bug.

• Builder Sidecar:

  • A specialized container that handles the "edit-compile-test" loop. It restores a build snapshot, applies a candidate patch, performs an incremental rebuild, and runs regression tests. This isolates build complexity from the CRS logic.

3. Key Contributions

1. Empirical Analysis: A systematic analysis of all seven AIxCC finalist codebases identifying the three specific barriers (duplication, lock-in, monolithic design) preventing adoption.
2. OSS-CRS Framework: A novel open-source framework providing a unified execution model, standard interface (libCRS), and budget-aware resource management (CPU, Memory, LLM).
3. Real-World Validation: Successfully ported the first-place system (ATLANTIS) to the new framework. The port required removing ~20 Azure VM dependencies and replacing Kubernetes orchestration with a flat Docker architecture.
4. Ensemble Capability: Demonstrated the ability to run multiple CRSs (including traditional fuzzers and LLM-based agents) simultaneously, exchanging artifacts to improve coverage and bug discovery.

4. Results

The authors evaluated OSS-CRS by running the ported ATLANTIS system against 8 OSS-Fuzz projects (spanning C/C++ and Java, ranging from 3k to 1.9M lines of code) on a single local machine (32 cores, 128GB RAM).

• Bug Discovery: Discovered 10 previously unknown bugs across the 8 projects.
  • Severity: 3 were classified as high severity.
  • Types: Included memory safety issues, logic bugs, undefined behavior (CWE-476, CWE-674, CWE-681), null-pointer dereferences, and schema validation errors.
  • Status: 3 bugs were fixed by upstream maintainers, 1 confirmed, and 6 pending review at the time of writing.
• Patch Generation: The system successfully synthesized patches for these bugs. Patches were validated by automatically rebuilding the target, re-running the Proof of Vulnerability (PoV) to confirm the crash was fixed, and running regression tests.
• Porting Efficiency:
  • Porting ATLANTIS-MULTILANG took ~3 person-days.
  • Porting ATLANTIS-C took ~4 person-days (due to tighter coupling with competition build pipelines).
  • Porting ATLANTIS-JAVA and the new CLAUDECODE (LLM patch generator) took ~1 person-day each.
  • Crucially, the core analysis algorithms remained untouched; only configuration, I/O adaptation, and build integration were modified.

5. Significance

• Democratization of CRS: OSS-CRS breaks the "cloud lock-in," allowing researchers and practitioners to run competition-grade autonomous security systems on local hardware or standard cloud environments without massive infrastructure overhead.
• Modularity and Composability: By breaking monolithic systems into composable components, the framework enables "ensemble" security. Researchers can now mix and match the best fuzzer from one team with the best patcher from another, facilitating controlled ablation studies and technique comparison.
• Actionable Security: Unlike many AI tools that only report potential bugs, OSS-CRS validates findings and generates patches, directly addressing the "CVE slop" problem where maintainers are overwhelmed by unverified reports.
• Cost Control: The built-in LLM budget management ensures that AI-driven security workflows are economically viable for organizations, preventing runaway API costs.
• Open Science: The framework, integrated CRSs, and evaluation artifacts are publicly available, fostering a community-driven approach to advancing autonomous cyber reasoning.

In conclusion, OSS-CRS transforms AIxCC from a closed competition into an open, reusable infrastructure, proving that autonomous vulnerability discovery and remediation can be effectively deployed against real-world open-source software.