Can AI Models Direct Each Other? Organizational Structure as a Probe into Training Limitations

1. Problem Statement

Current AI coding agents (e.g., SWE-agent, OpenHands) typically operate as monolithic agents: a single model handles the entire cognitive budget, from root cause analysis and planning to code execution and testing. While effective, this approach is costly when using high-end models for every step and lacks the hierarchical efficiency found in human software organizations.

The paper investigates a fundamental question: Can an expensive, high-capability AI model effectively direct a cheap, low-capability model to solve software engineering tasks?
The authors hypothesize that mimicking a human "Manager/Individual Contributor (IC)" structure could decouple expensive reasoning from cheap execution, but they also suspect that current models are trained as unified agents and may struggle with the specific roles of "directing" (delegation) and "following" (scoped execution).

2. Methodology: The MANAGERWORKER Pipeline

The authors introduce MANAGERWORKER, a two-agent pipeline designed to test organizational structures in AI.

Roles & Capabilities

• Manager (Expensive Model, Text-Only):
  • Role: Analyzes issues, dispatches exploration tasks, synthesizes findings into plans, reviews implementations, and provides feedback.
  • Constraint: No repository access. It cannot read files or run commands directly. It relies entirely on text reports from the worker. This forces abstract reasoning and prevents the model from reverting to its trained habit of "reading to conclude."
• Worker (Cheap Model, Full Access):
  • Role: Executes specific tasks (file exploration, code editing, running tests) under the manager's guidance.
  • Constraint: Receives tightly scoped instructions. It has full agentic access to the repository but lacks the high-level strategic reasoning of the manager.

Pipeline Phases

The system operates in iterative loops mirroring human workflows:

1. Analysis: Manager generates 2–3 focused exploration tasks.
2. Exploration Loop (Max 3 rounds): Workers execute tasks in parallel and report findings. The manager reviews reports and decides whether to explore further or proceed to planning.
3. Planning: Manager synthesizes findings into a specific implementation plan (exact code changes).
4. Implementation Loop (Max 3 rounds):
   • Round 1 (Guided): Worker implements with some autonomy to adapt to code reality.
   • Round 2+ (Strict): If the patch fails, the manager provides strict, corrective instructions. The worker must follow these exactly.

Experimental Setup

• Benchmark: 200 instances from SWE-bench Lite (real-world GitHub issues across Python libraries like Django, Flask, etc.).
• Models:
  • Strong: Claude Sonnet 4.6 (used as Manager and Strong Direct baseline).
  • Weak: GPT-5-mini (used as Worker and Weak Direct baseline).
• Configurations Tested:
  1. MANAGERWORKER: Strong Manager + Weak Worker (Full pipeline).
  2. Simple Loop: Strong Manager + Weak Worker (Minimal review loop, no structured exploration/planning).
  3. Strong Direct: Strong Model acting alone.
  4. Weak Direct: Weak Model acting alone.
  5. Weak→Weak: Weak Manager + Weak Worker (Testing structure without capability gap).

3. Key Contributions

1. The MANAGERWORKER Pipeline: A novel multi-agent system applying organizational hierarchy to SE tasks, proving that expensive reasoning can substitute for expensive execution.
2. Quality-Cost Pareto Analysis: Demonstrates that a Strong Manager directing a Weak Worker achieves 62% resolution, matching a Strong Single Agent (60%) while using ~5x fewer expensive tokens (shifting ~90% of tokens to the cheap model).
3. Ablation of Structure vs. Capability:
   • Shows that structure without a capability gap is harmful: A Weak Manager directing a Weak Worker (42%) performs worse than a Weak Agent alone (44%).
   • Shows that active direction > passive review: A "Simple Loop" (review only) achieves 53%, while the full structured pipeline achieves 62%, proving that planning and exploration direction are the primary value drivers.
4. Diagnosis of Training Limitations: Identifies that current models are trained as monolithic agents. Splitting them into director/worker roles fights their training distribution, requiring specific design workarounds (e.g., text-only managers, guided-then-strict prompting).

4. Key Results & Findings

• Finding 1 (Substitution): An expensive manager can effectively substitute for an expensive worker. The manager's value lies in reasoning (planning/review), not execution.
• Finding 2 (The Capability Gap): Organizational structure requires a genuine capability gap. If the manager is weak, the structure amplifies errors (plausible but wrong plans) rather than correcting them.
• Finding 3 (Active Direction): The value of the manager comes from directing (planning, exploration), not just reviewing. Structured phases add 11 percentage points over the baseline, while mere review adds only 2.
• Finding 4 (Training Mismatch): The pipeline succeeds by externalizing organizational logic into code, forcing models to stay close to their training distributions (Manager = Text Gen, Worker = Tool Use).

5. Significance & Implications

Theoretical Insight: The Training Distribution Problem

The paper argues that current LLMs are trained as monolithic agents (reason + execute in one session).

• Directors struggle because they are trained to read files and conclude; when forced to delegate without reading, they hallucinate or fail to abstract.
• Workers struggle because they are trained to be autonomous; when forced to follow strict instructions without reasoning, they drift or fail to execute.
• Weak Models lack the specific skill of "delegation" (writing actionable task briefs), which is distinct from code generation.

Practical Implications

• Cost Efficiency: Organizations can achieve premium results by using a "small brain" for strategy and a "cheap brain" for execution, provided the strategy brain is significantly stronger.
• Design Patterns: Multi-agent systems should externalize structure (phase transitions, loop limits, prompt switching) into the pipeline code rather than relying on the models to manage their own roles.
• Future Training Directions: The paper suggests specific training objectives to improve multi-agent systems:
  1. Delegation Training: Fine-tuning on task briefs, code reviews, and specifications.
  2. Scoped Execution Training: Training models to follow instructions without "editorializing" or over-thinking.
  3. Mode Switching: Training models to recognize when to explore freely vs. when to follow strict constraints.

Limitations

• The study is limited to Python repositories (SWE-bench Lite).
• The "Weak→Weak" failure suggests this architecture is not a universal fix; it strictly requires a high-capability director.
• The "Planning Gap" remains: Managers cannot plan for "unknown unknowns" that only emerge during implementation.

Conclusion

The paper concludes that yes, an expensive AI model can direct a cheap one, but only if the system is carefully designed to compensate for the models' training limitations. The "Manager/Worker" pattern works not because the models are naturally good at these roles, but because the pipeline forces them to operate in modes (text generation vs. tool use) that align with their training, while the code handles the organizational complexity. This reveals a critical gap in current AI training: the lack of skills in delegation and scoped execution.