SPOQ: Specialist Orchestrated Queuing for Multi-Agent Software Engineering

SPOQ is a novel multi-agent software engineering methodology that integrates wave-based topological dispatch, dual validation gates, and human-in-the-loop oversight to significantly reduce defects, eliminate planning cycles, and achieve substantial speedups while maintaining high code quality across diverse repositories.

The Gist

Imagine you are trying to build a massive, complex Lego castle. In the old way of doing things with AI, you might ask one robot to build the whole thing, or you might ask a team of robots where they all stand in a line, waiting for the person in front of them to finish before they can start. This is slow, and if the first robot makes a mistake, the whole line has to stop and fix it later.

The paper introduces SPOQ (Specialist Orchestrated Queuing), which is like a super-smart construction manager for a team of AI robots. Instead of making them wait in line or work alone, SPOQ organizes them to work together efficiently, checks their work constantly, and even brings in a human boss to help when things get tricky.

Here is how SPOQ works, broken down into simple parts:

1. The "Wave" System (No More Waiting in Line)

Imagine a stadium where the crowd does "the wave." Everyone in one section stands up at the same time, then the next section stands up, and so on. No one is waiting for the person next to them to finish; they just wait for the signal from the manager.

SPOQ does this with software tasks. It looks at a list of things that need to be built (like "build the login page" or "create the database") and draws a map of which ones depend on others.

• The Old Way: Robot A builds the login page, waits for Robot B to finish the database, then Robot C starts the chat feature.
• The SPOQ Way: The manager sees that the login page and the database don't need each other. So, Robot A and Robot B start at the exact same time (in the same "wave"). Only when they are both done does the next wave start.
• The Result: The paper claims this makes the work finish up to 14 times faster in ideal conditions, and still about 1.4 times faster even when the computers are busy.

2. The "Double-Check" Gates (Don't Build on a Bad Foundation)

Imagine building a house. If you don't check the blueprints before you start, you might build the kitchen in the wrong spot. If you don't check the walls after you build them, you might find a crack later.

SPOQ puts up two strict "gates" that the work must pass through:

• Gate 1 (Before Building): The AI team must write a plan. A "reviewer robot" checks this plan against a strict checklist (10 different rules, like "Is the goal clear?" and "Are the steps logical?"). If the plan scores below 95%, they have to rewrite it before writing a single line of code. This stops mistakes before they happen.
• Gate 2 (After Building): Once the code is written, another robot checks it against a different checklist (10 rules like "Does it pass the tests?" and "Is it secure?"). If it fails, it gets sent back to be fixed immediately.

The paper found that using these two gates reduced the number of bugs (defects) by more than half and made the final software pass almost every single test (99.75%).

3. The "Human-as-Agent" (The Human Boss in the Loop)

In many AI systems, humans just watch from the sidelines. In SPOQ, the human is an active member of the team, like a senior architect who is part of the crew.

• Before the work starts: The human helps break the big project into small, manageable pieces and checks the plan.
• During the work: If the AI robots get stuck or confused, they can pause and ask the human for help.
• The Result: When a human helps plan the project, the final result is even better. The paper shows that with human help, the number of remaining bugs dropped to almost zero (0.03 bugs per task), and the software passed tests 99.75% of the time.

4. The "Three-Tier" Robot Team (Right Tool for the Right Job)

SPOQ doesn't use the same expensive, slow robot for every job. It uses a smart mix of three types of robots:

• The "Opus" (The Master Builder): This is the most powerful (and expensive) robot. It does the hard, complex coding work.
• The "Sonnet" (The Quality Inspector): This is a balanced robot. It checks the Master Builder's work to make sure it's good.
• The "Haiku" (The Quick Fixer): This is a fast, cheap robot. It looks at error messages to figure out why something broke so the team can fix it quickly.

By using the right robot for the right job, the system saves money while keeping quality high.

What the Paper Actually Proved

The authors tested this system in a few ways:

• Speed Tests: They gave the system fake tasks to see how fast it could organize them. SPOQ was much faster than systems that make robots wait in line.
• Quality Tests: They compared SPOQ to standard AI coding tools. SPOQ made fewer mistakes, had better plans, and wrote code that passed more tests.
• Real-World Use: They used SPOQ on 17 different real software projects (like websites and data tools). They completed over 1,800 tasks and ran nearly 14,000 tests, with a 99.87% pass rate.

In short: SPOQ is a new way to organize AI robots to build software. It uses a "wave" system to let them work in parallel, puts up strict checkpoints to catch errors early, and keeps a human in the loop to guide the team. The result is software that is built faster, has fewer bugs, and is more reliable.

Technical Summary

Technical Summary: SPOQ (Specialist Orchestrated Queuing) for Multi-Agent Software Engineering

1. Problem Statement

While multi-agent AI systems show promise for automating software engineering, existing approaches suffer from three fundamental limitations:

1. Coordination Overhead: Systems like ChatDev and MetaGPT rely on sequential role-playing or message-passing, creating bottlenecks that prevent the realization of parallel execution speedups.
2. Quality Control Gaps: Most systems lack structured validation between planning and execution. Agents often execute flawed plans without rigorous assessment, leading to wasted computation, and post-execution quality checks are often informal or absent.
3. Limited Human Oversight: Fully autonomous systems exclude human judgment, missing opportunities to leverage human expertise for task decomposition, ambiguity resolution, and quality assessment.

2. Methodology: The SPOQ Framework

SPOQ (Specialist Orchestrated Queuing) addresses these challenges through a four-stage pipeline (Epic Planning, Epic Validation, Agent Execution, Agent Validation) built on three core innovations:

A. Wave-Based Topological Dispatch

SPOQ models task dependencies as a Directed Acyclic Graph (DAG). Using topological sorting, it computes execution waves—groups of independent tasks that can be executed in parallel.

• Mechanism: Tasks within the same wave execute concurrently, while waves execute sequentially to respect dependencies.
• Goal: Maximize parallelism without coordination overhead, approaching the theoretical critical-path lower bound.

B. Dual Validation Gates

SPOQ enforces quality through two structured checkpoints with explicit metrics (10 metrics each) and quantified thresholds:

1. Planning Validation (Pre-Execution): Assesses the epic plan against 10 metrics (e.g., Vision Clarity, Dependency Graph, Coverage Completeness). A 95% aggregate threshold (with a 90% minimum per metric) ensures plans are structurally sound before agents are spawned.
2. Code Validation (Post-Execution): Assesses completed code against 10 metrics (e.g., Syntactic Correctness, Test Pass Rate, SOLID Adherence). A 95% aggregate threshold (with an 80% minimum per metric) ensures code quality before acceptance.

• Cascade Effect: If any individual task fails validation, the entire epic's score is capped, preventing "carrying" weak tasks on the strength of strong planning.

C. Human-as-an-Agent (HaaA)

SPOQ treats the human specialist not as a passive observer but as an active, bidirectional agent within the loop:

• Human $\to$ System: Humans participate in epic planning, validate plans, and can intervene during execution.
• System $\to$ Human: Agents can explicitly request human assistance when facing ambiguity, blocked progress, or decisions beyond their scope.
• Role: The human acts as a high-value agent for task decomposition and validation, amplifying the system's output quality.

D. Three-Tier Agent Hierarchy

To optimize cost-quality tradeoffs, SPOQ employs a tiered agent structure:

• Opus Workers: High-capability, high-cost agents for task execution.
• Sonnet Reviewers: Balanced capability/cost agents for quality assurance and validation.
• Haiku Investigators: Low-cost, fast-response agents for build failure triage.
• Note: While the reference implementation uses Anthropic's Claude family, the methodology is platform-agnostic and can map to other providers (e.g., GPT-4, Gemini, Qwen).

3. Key Contributions

The paper makes the following contributions:

1. Formal Framework: A wave-based orchestration method computing parallel execution waves from task dependency graphs.
2. Agent Hierarchy: A three-tier model (Opus/Sonnet/Haiku) optimizing cost vs. capability.
3. HaaA Paradigm: A structured bidirectional collaboration model for human-AI task decomposition.
4. Dual Validation System: Explicit metrics and thresholds for both planning and code quality.
5. Controlled Benchmarks: A suite testing scheduling efficiency, planning quality, validation effectiveness, and human-AI collaboration.
6. Cross-Provider Replication: Validation of results using a locally hosted open-weights model (Qwen3.6-35B-A3B) to prove gains stem from orchestration, not specific model capabilities.
7. Longitudinal Deployment: A field study across 17 repositories, 8,589 commits, and 1,822 completed tasks.

4. Experimental Results

Experiment 1: Scheduling Efficiency

• Unbounded Synthetic DAGs: Wave dispatch approached the critical-path lower bound with a ratio of 1.03–1.11, achieving speedups up to 14.3× over sequential execution.
• Hardware-Bounded (2-slot local backend): Delivered a stable 1.4× speedup, matching the hardware concurrency ceiling.
• Replication: Results held across Qwen3.6-35B-A3B, confirming the algorithmic nature of the gains.

Experiment 2: Planning Quality

• Coverage: Structured SPOQ planning improved requirement coverage from 93.0% to 99.75%.
• Errors: Eliminated cyclic plans entirely (0/4 vs. 3/4 in baseline) and reduced dependency errors.
• Parallelism: Increased parallelism potential from 31.0 to 75.25.
• Cross-Provider: On the local Qwen model, SPOQ recovered 35 points of coverage and 52.5 points of parallelism compared to the unaided baseline, eliminating cyclic plan failures.

Experiment 3: Validation Effectiveness

• Defects: Dual validation reduced defects per task from 0.34 to 0.20.
• Test Pass Rate: Increased from 91.25% to 99.75%.
• Rework: Reduced rework cycles from 3.75 to 1.00 per task.
• Static Analysis: Eliminated static analysis warnings (0.00) under Full SPOQ.
• Security: Identified more latent security issues (4.75 vs. 1.75), indicating broader detection coverage rather than weaker security.

Experiment 4: Human-as-Agent (HaaA)

• Defects: Human-assisted planning reduced residual defects from 0.47 to 0.03 per task.
• Pass Rate: Increased test pass rate from 96.5% to 99.75%.
• Trade-off: While rework cycles increased (indicating more thorough correction), the final system quality was significantly higher.
• Planning Quality: Human review improved coverage (88.75% $\to$ 95.00%) and reduced dependency errors even before execution.

Field Deployment Study

• Scale: Deployed across 17 repositories with 1,822 completed tasks and 13,866 executed tests.
• Success Rate: Achieved an aggregate test pass rate of 99.87%.
• Adoption: Includes third-party adoption (e.g., Adrata's speedrun-gitlab), demonstrating transferability beyond the originating team.

5. Significance and Claims

The paper positions SPOQ as a step toward AI-native software engineering, where processes are designed around AI capabilities rather than retrofitting AI into human workflows.

• Orchestration over Model Capability: The primary claim is that the observed improvements (speedup, quality, reliability) stem from the orchestration methodology (wave dispatch, dual validation, HaaA) rather than the specific LLM used. This is supported by consistent gains across both frontier models (Claude) and local open-weights models (Qwen).
• Human-AI Collaboration: SPOQ demonstrates that treating humans as active agents (HaaA) significantly reduces residual defects and improves final system robustness, challenging the notion of fully autonomous agents.
• Quality as a Constraint: By enforcing rigorous validation gates, SPOQ shifts defect detection earlier in the pipeline, reducing downstream rework and improving overall system quality.
• Scalability: The methodology enables a single human specialist to direct a digital workforce, achieving throughput (75–150 tasks/day) previously requiring 8–10 engineers.

The authors acknowledge limitations, including the upfront investment in planning, dependency on human specialist skill, and the need for broader independent replication. However, the combination of controlled benchmarks and longitudinal field evidence suggests SPOQ offers a viable, scalable framework for multi-agent software development.