Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting

Technical Summary: Stop Hand-Holding Your Coding Agent: Engineering the Loops that Replace Step-by-Step Prompting

Problem Statement
The paper addresses a shift in the discourse surrounding coding agents (circa mid-2026), where practitioners moved from step-by-step prompting to "designing the loop that prompts the agent." However, this practice arrived through informal channels (blogs, talks) without a rigorous definition, leading to conceptual confusion. The core problem is the lack of a clear distinction between three distinct concepts often conflated under the term "loop":

1. Ordinary programming loops: Standard control flow structures.
2. Internal agent cycles: The inherent perceive-act-observe machinery provided by the agent harness (e.g., ReAct frameworks), which exists regardless of human design.
3. Loop specifications: The new object of study—an external, bounded, reusable artifact designed by humans to govern how an agent pursues a goal autonomously.

The paper argues that without a formal definition and taxonomy, the practice risks conflating the "engine" (the harness) with the "pilot" (the human-designed loop), leading to fragile systems that may suffer from reward hacking, specification gaming, or cognitive surrender.

Methodology
This work is a position paper anchored in a descriptive corpus study, not a controlled experiment or a proposal for new algorithms. The methodology consists of four main components:

1. Conceptual Definition: Defining the "loop specification" and distinguishing it from existing agent architectures and programming constructs.
2. Taxonomy Construction: Developing a five-dimensional framework for analyzing loop specifications (Trigger, Goal Type, Verification Ladder, Architecture, Terminal States).
3. Corpus Analysis: Manually coding and analyzing 50 real-world loops from the public "Loop Library." The authors recorded specific attributes for each loop to determine current industry practices.
4. Literature Grounding: Mapping observed practices and design principles against scientific literature regarding self-correction, reward hacking, and the fragility of "model-as-judge" systems.
5. Artifact Development: Creating and releasing sandeco-loop, an open skill (tool) that operationalizes the proposed design principles into a repeatable authoring procedure.

Key Contributions

1. Definition and Scope of Loop Engineering
The paper defines a loop specification as a bounded, reusable artifact comprising five parts:

• Trigger: Manual, scheduled, or event-based initiation.
• Goal: Preferably verifiable via deterministic checks or rules.
• Execution: The agent working, ideally via named, proven skills.
• Verification: A check that validates the result.
• Stopping Rule: A mechanism to reach a named terminal state (e.g., success, blocked, exhausted) without mistaking errors for success.
• Memory: Persistent state stored externally (on disk) rather than in the conversation window.

The authors position loop engineering as a new layer in the progression from Prompt → Context → Harness → Loop. They argue that loop engineering does not retire prompt engineering but complements it; a loop is essentially a prompt repeated with scaffolding.

2. Anatomy and Taxonomy
The paper introduces a structured taxonomy for loop design:

• Verification Ladder: A five-level hierarchy of rigor:
  • Level 1: Deterministic (assertions, exit codes).
  • Level 2: Rule-based (linters, schemas).
  • Level 3: Delayed field truth (tests, deployment, customer response).
  • Level 4: Model-as-judge (scoring by rubric).
  • Level 5: Human checkpoint.
  • Key Insight: Levels 1 and 2 constitute the "autonomous zone." The paper warns against treating Level 4 (model opinion) as Level 1 (ground truth).
• Architecture: Distinguishes between solo agents, maker-checker splits (where the generator is distinct from the verifier), and multi-agent orchestration.
• Terminal States: Emphasizes the necessity of naming states (e.g., stalled, blocked) to prevent infinite loops or false positives.

3. Empirical Findings from the Loop Library
Analysis of the 50-loop corpus reveals a "maturity mismatch" in current practice:

• Strengths: The practice has matured in areas critical for reliability. 70% of loops verify in the autonomous zone (Levels 1–2), 74% name their terminal states, and 66% set verifiable goals.
• Weaknesses: The practice is immature in areas required for true autonomy. Only 22% use automated triggers, 20% call named reusable skills, and 32% utilize persistent memory.
• Implication: Current loops solve "how do I know it is done" better than "how does this run without me."

4. Design Principles and Anti-Patterns
The paper distills principles grounded in literature:

• Define "Done" First: Use frozen, external yardsticks rather than self-assessment.
• Act Without Breaking: Make small, reversible changes; keep the "maker" distinct from the "checker" to prevent reward hacking.
• Sustain the Loop: Persist state on disk; curate memory rather than blindly accumulating it.
• Anti-Patterns Identified:
  • The "While-True" around a Stranger: Unbounded retries without skills or checks.
  • Self-Approving Loop: The same model generating and grading work (leads to reward hacking).
  • Specification Gaming: Optimizing for the check rather than the task.
  • Pretending Level 4 is Level 1: Treating model opinion as deterministic truth.
  • The Unattended Runaway: Loops without stopping rules or budget ceilings.

5. The sandeco-loop Skill
As a concrete deliverable, the authors released sandeco-loop, a tool that guides users through a triage process (determining if a task actually requires a loop), conducts an interview to gather design elements, and emits a hardened specification document. It enforces the principles of external verification, named terminal states, and cost tracking.

Results and Significance
The paper does not claim to have run a benchmark or measured performance improvements. Instead, its significance lies in:

1. Vocabulary and Rigor: Providing a precise vocabulary to distinguish the "loop specification" from internal agent cycles, allowing the community to examine claims rather than merely repeat slogans.
2. Evidence-Based Practice: Demonstrating that the community has naturally converged on robust verification (autonomous zone) but lags in automation (triggers/memory).
3. Sobering the Discourse: Challenging the "death of prompt engineering" narrative. The paper argues that loops are a tool for prompting, not a replacement, and that the human role shifts from step-by-step instruction to designing the system that governs iteration.
4. Identifying Limits: Highlighting that loops do not solve the "comprehension debt" (humans understanding code they didn't write) or the "verification burden" (the cost of checking agent output). It explicitly warns against the "cognitive surrender" of trusting loops that appear to cope without human oversight.

Conclusion
The paper concludes that the loop specification is a distinct, real object worthy of deliberate design. The central skill of this new discipline is the check, not the prompt. The authors posit that the next phase of engineering should focus on the underdeveloped areas of the corpus: automated triggering, reusable skills, and durable memory, while maintaining a strict adherence to verification rigor to avoid the pitfalls of reward hacking and specification gaming. The ultimate metric for success, they propose, is "cost per accepted change," a metric currently unmeasured but essential for evaluating the economic viability of autonomous loops.