Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Search

Vulcan is a framework that leverages LLMs to synthesize safe, instance-specialized systems heuristics by isolating decision logic in a restricted language (Anvil) and providing trusted runtime abstractions, achieving significant performance improvements in spot-VM scheduling, cache eviction, and tiered-memory systems while guaranteeing execution safety.

The Gist

Imagine you are the manager of a massive, high-tech warehouse. Your job is to decide which items to keep on the fast, expensive shelves (like a CPU cache or fast memory) and which to move to the slow, cheap basement (like a hard drive or spot cloud instances).

For decades, warehouse managers have relied on hand-written rulebooks. These rulebooks are smart, but they are rigid. They were written for a specific type of warehouse with a specific mix of products. If you change the building, the products, or the customers, the old rulebook often fails. Writing a new one for every single scenario is slow, expensive, and impossible to scale.

Recently, people tried using AI (Neural Networks) to learn new rules. But these AI "black boxes" are hard to trust. You can't easily see why they made a decision, and if they crash the warehouse, it's a disaster.

Then came Large Language Models (LLMs), the AI that writes code. The idea was: "Let's ask the AI to write the rulebook for us!" But this had a big problem. If you let the AI write the entire rulebook, it might write code that crashes the system, leaks memory, or breaks the warehouse's safety locks. It's like giving a toddler a full set of power tools and saying, "Build me a house." They might build something cool, but they'll likely cut their finger or burn the place down.

Vulcan is a new framework that solves this by changing how we ask the AI to help. Think of it as building a safe, specialized workshop for the AI.

The Three Pillars of Vulcan

1. The "Chef and Sous-Chef" Division of Labor

In a restaurant, the Head Chef (the human developer) knows the kitchen layout, the safety codes, and how to handle the dangerous knives. The Sous-Chef (the AI) is great at inventing new flavor combinations.

Vulcan splits the work:

• The Human (Chef): Handles the "heavy lifting." They build the kitchen, manage the inventory (state), and ensure the doors are locked. They define the structure of the problem.
• The AI (Sous-Chef): Is only allowed to write the decision logic. They get a list of ingredients (data) and must write a simple recipe: "If the soup is hot, add salt; if it's cold, add pepper." They are not allowed to touch the stove, move the furniture, or manage the inventory.

This means the AI can focus on being creative with the rules without accidentally breaking the kitchen.

2. The "Magic Dashboard" (Listeners)

Usually, to make a smart decision, you need to know things like "What was the average temperature over the last hour?" or "How many times has this item been picked in the last 5 minutes?"

If the AI had to write the code to track these numbers, it would likely make mistakes (like forgetting to reset a counter).

Vulcan provides a Magic Dashboard (called libVulcan).

• The human developer sets up the dashboard with pre-built gauges (Listeners).
• The AI just looks at the dashboard. It doesn't need to know how the gauge works; it just reads the number.
• This ensures the AI gets rich, accurate data without having to write complex, error-prone code to track it.

3. The "Training Wheels" Language (Anvil)

Even if the AI is only writing a simple recipe, it could still write a loop that runs forever or tries to use more memory than exists.

Vulcan forces the AI to write in a special, restricted language called Anvil.

• No dangerous tools: The AI cannot use pointers (which can crash things), cannot allocate unlimited memory, and cannot write loops that never stop.
• Safety by construction: Because the language physically prevents these dangerous actions, any code the AI writes is guaranteed to be safe. It's like giving the AI a toy hammer instead of a sledgehammer. It can still build a house, but it can't accidentally knock the roof off.

The Results: Does it Work?

The researchers tested Vulcan in three real-world "warehouse" scenarios:

1. Cloud Spot VMs (The "Bargain Bin" Scheduler):

   • The Problem: Cloud providers offer cheap computers that can be taken away at any moment. You need a rule to decide when to use the cheap ones and when to switch to expensive, reliable ones.
   • The Result: Vulcan created a rule that saved 4.9 times more money than the best human-written rule. It learned to spot patterns in availability that humans missed.

2. Cache Eviction (The "Shelf Cleaner"):

   • The Problem: When a web cache is full, which item should you throw out to make room for a new one?
   • The Result: For specific types of websites, Vulcan's rules reduced the "miss rate" (when a user has to wait for data) by 2 times compared to standard rules. It found that different websites need different "cleaning" strategies.

3. Memory Tiering (The "Fast vs. Slow Shelf"):

   • The Problem: Moving data between fast memory and slow memory to keep apps running smoothly.
   • The Result: Vulcan's rules improved application performance by 10% compared to existing systems, and these rules worked even when tested on real hardware they hadn't seen before.

The Bottom Line

Vulcan doesn't try to replace the human expert. Instead, it creates a safe sandbox where the AI can do what it's good at (finding the perfect pattern or rule) while the human does what they're good at (keeping the system safe and structured).

By restricting the AI to a small, safe job and giving it a dashboard of pre-made tools, Vulcan manages to create computer rules that are safer, faster, and smarter than what humans can write alone, without the risk of the AI crashing the system.

Technical Summary

Technical Summary: Vulcan: Instance-specialized, Verifiable Systems

Problem Statement

Systems resource management has traditionally relied on manually designed heuristics. However, the increasing heterogeneity of hardware, diversity of workloads, and variability in deployment environments have made the manual design of instance-specialized heuristics (tailored to specific combinations of hardware, workload, and objectives) expensive and unscalable. While neural models offered a path to specialization, they suffer from opacity, operational complexity, and integration difficulties.

Recent advances in Large Language Models (LLMs) suggest a new approach: synthesizing executable code directly. However, naive application of LLMs to systems code faces three critical challenges:

1. Performance: Generated heuristics must match or exceed human-designed baselines.
2. Execution Safety: Generated code must be free from crashes, memory errors, leaks, and non-termination.
3. System Integration: Code must correctly interact with existing system APIs and state management mechanisms.

Unconstrained LLM synthesis often produces performant heuristics but fails on safety and integration (e.g., memory leaks, API violations). Conversely, overly constrained synthesis preserves safety but limits the policy space, resulting in negligible performance gains over existing baselines.

Methodology: The Vulcan Framework

Vulcan addresses these challenges by refactoring systems heuristics around a synthesis boundary that isolates core decision logic from state management and system interaction. The framework relies on two key ideas:

1. Decoupling Decision Logic from State Management (libVulcan)

Vulcan restricts LLM-generated code to stateless decision functions while delegating state management to trusted, developer-maintained code. This is achieved through:

• Rank and Value Interfaces: The core decision logic of most resource-management tasks is expressed through two interfaces:
  • Value: Computes a scalar control variable (e.g., congestion window size, frequency) based on system state features.
  • Rank: Scores a set of candidate objects (e.g., cache entries, threads) to determine an ordering or selection.
• Listener Abstractions (libVulcan): To enable meaningful policy exploration without requiring LLMs to manage data structures, Vulcan provides a library of "listeners." These modular blocks attach to raw signals to compute derived statistics (e.g., moving averages, percentiles, population means) on the fly. The LLM queries these listeners but does not implement the underlying state management, eliminating a major source of integration and safety bugs.

2. Safety by Construction via Anvil (DSL)

To guarantee execution safety, Vulcan synthesizes heuristics in Anvil, a restricted Domain-Specific Language (DSL).

• Restrictions: Anvil excludes heap allocation, pointers, recursion, and unbounded loops.
• Guarantees: By construction, any well-formed Anvil program guarantees:
  • Memory Safety: No out-of-bounds access, use-after-free, or double-frees.
  • Leak Freedom: No unbounded memory growth.
  • Termination: No infinite loops.
• Compilation: Anvil code is parsed against its grammar and compiled to C, which is then processed by standard toolchains. The DSL supports extern declarations for C/C++ functions (like listener APIs), ensuring safety properties extend transitively to the synthesized code.

Workflow

1. Developer Input: Developers provide a heuristic template (defining raw features and triggers), an evaluator, and a natural language prompt describing the task.
2. Synthesis Loop: A code-generation framework (e.g., OpenEvolve, ShinkaEvolve) iteratively generates candidate heuristics in Anvil.
3. Evaluation: Candidates are scored by the evaluator (often using trace-driven simulation or short workloads to reduce cost).
4. Selection: The best-scoring heuristic is returned as the final instance-specialized policy.

Key Contributions

1. Framework Design: Identification of a synthesis boundary that isolates core decision logic, enabling LLM-driven search while retaining system-facing structure in trusted code.
2. libVulcan: A runtime library providing Rank and Value interfaces and a suite of listener abstractions for deriving rich statistics, serving as a common substrate for specialized heuristics.
3. Anvil: A restricted policy language that guarantees execution safety properties (memory safety, leak freedom, termination) by construction, removing the need for manual safety review of synthesized code.
4. Empirical Evaluation: Demonstration that Vulcan can synthesize heuristics that are competitive with or superior to human-designed baselines across three distinct domains while satisfying safety constraints.

Results

The paper evaluates Vulcan across three well-studied resource management domains:

1. Spot VM Scheduling:

   • Task: Deciding when to switch between on-demand and spot instances to minimize cost while meeting deadlines.
   • Result: Vulcan synthesized a heuristic that achieved 4.9× higher savings compared to the state-of-the-art "Uniform Progress" baseline and 1.7× higher savings compared to an unconstrained LLM synthesis approach (ADRS).
   • Safety: All synthesized code satisfied execution safety properties by construction.

2. Cache Eviction:

   • Task: Selecting objects to evict to maximize hit rates across diverse traces and cache sizes.
   • Result: For specific instances, Vulcan synthesized specialized heuristics that improved miss ratios by up to 2× compared to baselines (e.g., FIFO, LRU, GDSF). Across 32 instances, Vulcan outperformed all baselines on 6 and was within 5% of the best on 13 more.
   • Safety: No execution safety violations were observed.

3. Tiered Memory Systems:

   • Task: Deciding which pages to promote from slow memory (e.g., CXL) to fast DRAM.
   • Result: A heuristic synthesized on an emulator generalized to real CXL hardware, yielding up to 10% higher application performance (goodput) compared to the ARMS baseline.
   • Generalization: Heuristics trained on specific workloads (e.g., GUPS) performed well on unseen workloads (e.g., graph analytics, databases) without further tuning.

Significance and Claims

The paper claims that the primary barrier to automated heuristic synthesis is not the LLM's ability to find performant policies, but the design of the synthesis boundary. By narrowing the search space to expressive yet constrained interfaces (Rank/Value) and enforcing safety through a restricted DSL (Anvil), Vulcan makes instance-specialized heuristic synthesis practical and safe.

The authors emphasize that Vulcan does not replace human expertise; rather, it leverages domain knowledge to define the structure (triggers, raw signals, mechanisms) while using LLMs to explore and refine the policy logic within that structure. This approach allows for the deployment of verified, high-performance heuristics that adapt to specific instances without the operational overhead or safety risks associated with neural models or unconstrained code generation.