Technical Summary: Runtime-Structured Task Decomposition for Agentic Coding Systems

Problem Statement

Current agentic coding systems (e.g., those built on AutoGen, LangGraph, and Claude Code) frequently rely on monolithic prompts to encode task logic, control flow, and output generation. While recent frameworks have introduced programmatic abstractions for orchestration, their task decomposition strategies are typically static, meaning the task structure is fixed at design time.

This architectural approach leads to three primary production engineering challenges:

1. Brittle Behavior: Systems struggle to adapt when intermediate outputs are invalid or when branching conditions emerge at runtime.
2. Poor Debuggability: Failures are difficult to isolate within a single, large prompt context.
3. Expensive Recovery: When a step fails, the system often requires a full-pipeline rerun or the re-execution of all downstream subtasks, resulting in high token costs and latency.

The central question addressed is whether task decomposition alone reduces recovery costs, or if runtime branching based on validated intermediate signals is the necessary mechanism for efficiency.

Methodology

The authors propose Runtime-Structured Task Decomposition (RSTD), an architectural pattern that externalizes task partitioning decisions from static prompt text into executable control flow.

Core Architecture

The system separates orchestration, state management, and LLM inference into three distinct layers:

1. Decomposition Engine: Governs runtime task partitioning using developer-authored conditional logic. Branching decisions are resolved dynamically based on runtime signals:
   • Schema validation outcomes.
   • Output content (e.g., non-empty results or confidence thresholds).
   • Subtask completion status.
2. Judgment Operators: LLM calls are scoped to single reasoning tasks with explicit, schema-validated output requirements. If validation fails, the system triggers a targeted repair prompt rather than a full pipeline rerun.
3. State Manager: Persists validated intermediate results keyed by subtask identifier. Downstream subtasks access prior state explicitly by key, ensuring bounded per-call context.

Experimental Setup

The authors evaluated RSTD against two baselines across two software engineering workloads (10 runs each):

• Monolithic: All task logic encoded in a single prompt.
• Static Decomposition: The same subtask graph as RSTD, but executed sequentially without runtime branching (fixed execution).
• RSTD: The proposed approach with runtime-controlled branching and selective retry.

Workloads:

1. Multi-File Code Debugging: A Python pipeline with three causally ordered bugs across two files.
2. Kubernetes Root Cause Analysis (RCA): Diagnosing an OOMKill event involving log parsing, anomaly detection, and remediation planning.

Metrics: Token consumption (retry cost), latency, and correctness. Natural schema validation failure rates were low (0–2%), so "retry cost" was measured under simulated failure (structurally malformed upstream input) to isolate architectural recovery costs.

Key Results

1. Static Decomposition Does Not Guarantee Cost Reduction

Contrary to the assumption that breaking tasks down is inherently more efficient, the Static Decomposition baseline often increased retry costs compared to the Monolithic baseline.

• Kubernetes RCA: Static retry cost was 1,632 ± 145 tokens, exceeding the Monolithic baseline (904 ± 17 tokens) by 80.5%. This occurred because fixed sequential execution forced the rerun of all downstream subtasks (3, 4, and 5) upon a failure in subtask 3.
• Multi-File Debugging: Static retry cost (933 tokens) was 32.7% higher than Monolithic (703 tokens) due to cascading re-execution of downstream tasks.

2. RSTD Achieves Significant Recovery Savings

By retrying only the failed subtask while preserving valid upstream state, RSTD drastically reduced recovery costs:

• Kubernetes RCA: RSTD retry cost dropped to 436 ± 132 tokens, a 51.7% reduction over Monolithic and a 73.2% reduction over Static Decomposition.
• Multi-File Debugging: RSTD retry cost was 460 tokens, a 34.6% reduction over Monolithic.

3. Overhead and Correctness

• Correctness: All configurations achieved 100% correctness across all 10 runs for both workloads.
• Baseline Overhead: RSTD incurs a higher baseline token cost during successful execution (e.g., 2,225 tokens vs. 703 tokens in debugging) due to multiple API calls and framework overhead.
• Latency: Framework overhead accounted for approximately 18% of total wall-clock latency. The latency gap between Monolithic and decomposed agents was primarily driven by additional API round-trips rather than orchestration inefficiency.

Significance and Claims

The paper claims that runtime branching, rather than decomposition structure alone, is the primary mechanism for reducing recovery costs in agentic systems.

• Architectural Distinction: RSTD is distinguished from existing frameworks (DSPy, LangGraph, AutoGen) by externalizing control flow. Unlike DSPy Assertions, which optimize prompts within a module, RSTD prevents malformed outputs from ever reaching downstream subtasks by gating state transitions.
• Operational Implications:
  • Selective Retry: Enables cost-efficient recovery by isolating failures to individual subtasks.
  • Targeted AgentOps: Subtask boundaries provide natural instrumentation points for monitoring and debugging.
  • Model Substitution: The architecture allows different models to be assigned to specific subtasks without reconfiguring the orchestration layer.

The authors conclude that while RSTD introduces a fixed overhead for successful runs, it is the superior pattern for production scenarios where failure rates are non-trivial or where subtasks involve expensive context re-ingestion. The study highlights that simply decomposing tasks statically is insufficient; dynamic, validation-gated control flow is required to prevent cascading re-execution costs.