🤖 AI

Operating-Layer Controls for Onchain Language-Model Agents Under Real Capital

This paper demonstrates that reliable autonomous language-model agents managing real capital in onchain markets depend less on base model capabilities and more on a robust operating layer featuring typed controls, policy validation, and execution guards, which collectively reduced critical failures like fabricated trading rules and enabled high settlement success across a large-scale 21-day deployment.

Original authors: T. J. Barton, Chris Constantakis, Patti Hauseman, Annie Mous, Alaska Hoffman, Brian Bergeron, Hunter Goodreau

Published 2026-04-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: T. J. Barton, Chris Constantakis, Patti Hauseman, Annie Mous, Alaska Hoffman, Brian Bergeron, Hunter Goodreau

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a highly intelligent, but slightly literal-minded, robot chef to run a restaurant. You give the chef a menu and a budget (your "mandate"), and you tell them, "Make good food and don't waste money."

In the past, researchers tested these robot chefs in a simulation: they gave them a fake menu and fake money. The chefs looked great on paper. But this paper describes what happened when they actually opened the restaurant with real money and real customers for 21 days.

Here is the story of what they found, using simple analogies.

The Big Discovery: It's Not About the Chef's Brain

The team expected that if they just hired a "smarter" robot chef (a more powerful AI model), the restaurant would run perfectly. They were wrong.

They found that the reliability of the restaurant didn't come from the chef's brain alone. It came from the kitchen management system surrounding the chef. They call this the "Operating Layer."

Think of it like this:

The Model (The Chef): Can read a recipe and chop vegetables.
The Operating Layer (The Kitchen Manager): Checks if the ingredients are fresh, stops the chef from throwing food away, ensures the chef doesn't accidentally set the kitchen on fire, and makes sure the bill matches what the customer ordered.

The paper argues that for an AI to handle real money, you need a super-strict Kitchen Manager, not just a smarter Chef.

The Experiment: "DX Terminal Pro"

The team set up a real-world test called DX Terminal Pro.

The Setup: 3,505 people put their own real money (ETH) into "vaults."
The Rule: Humans could set the rules (like "be risky" or "be safe"), but they couldn't touch the money directly. Only the AI agents could buy and sell tokens.
The Stakes: Every time the AI made a trade, it cost real money in fees (2.3% per trade). If the AI made a mistake, real money vanished.
The Scale: The AI made 7.5 million decisions, trading about $20 million worth of crypto.

The "Kitchen Nightmares" (Failure Modes)

Before the real event, the team ran tests and found five specific ways the AI chefs were messing up. They fixed these by changing the "Kitchen Manager" rules, not by firing the chefs.

The "Fake Rule" Problem (Rule Fabrication):
- The Glitch: The AI would invent rules that didn't exist, like saying, "I must sell because 'Rule #2' says so," when no such rule existed.
- The Fix: They told the AI, "Don't make up laws. If you didn't see it written down, it's not a rule."
- Result: Fake rules dropped from 57% of decisions to just 3%.
The "Fee Paralysis" Problem:
- The Glitch: The AI saw the 2.3% fee and got so scared it refused to trade at all, even when it was a good opportunity. It was like a chef refusing to cook because the stove costs money to turn on.
- The Fix: They moved the fee warning to the end of the instructions, right after the market opportunity. They taught the AI: "Yes, there is a fee, but here is why the trade is worth it."
- Result: The AI stopped freezing up and started trading again.
The "Math Confusion" Problem (Number Hardening):
- The Glitch: The AI treated soft suggestions like hard laws. If you said "try to spend about 33%," the AI would obsessively try to hit exactly 33% every single time, even when it made no sense.
- The Fix: They stopped using exact numbers and used comparisons instead (e.g., "spend more when the market is hot").
- Result: The AI started acting more naturally and less robotically.
The "Token Trap" Problem (Tokenomics Misread):
- The Glitch: One token crashed in price. The AI saw the crash and immediately sold everything, not realizing that the token had a special rule where holders get paid even if the price drops. It was like selling a coupon just because the store changed its logo, not realizing the coupon still works.
- The Fix: They gave the AI a clear, structured explanation of the special payment rule before showing the price crash.
- Result: The AI stopped panicking and made smarter decisions.
The "Clock Watcher" Problem (Cadence Trading):
- The Glitch: The AI started trading just because "it's been 10 minutes since the last trade," ignoring the market. It was like a chef chopping vegetables just because the clock said it was time, not because the ingredients needed chopping.
- The Fix: They told the AI to ignore the clock and only trade based on market signals.

The Results: A Better Restaurant

Once they fixed the "Kitchen Manager" (the Operating Layer), the system worked incredibly well:

99.9% Success Rate: Almost every trade the AI tried to make was valid and went through.
Real Behavior: The AI didn't just follow orders; it reacted to the market. Sometimes, when one AI saw a good deal, hundreds of others saw it too and bought at the same time (a "herd" effect), just like real humans do.
Better Instructions: Users who gave clear, specific instructions (using sliders and checkboxes) got better results than those who just wrote vague notes like "make me rich."

The Main Takeaway

The paper concludes that if you want an AI to handle real money, don't just look for a smarter AI model.

You need to build a better system around it. You need to:

Translate human wishes into clear, structured rules.
Check the AI's work before it spends a dime.
Keep a perfect record of every thought, decision, and trade so you can figure out exactly what went wrong if it does.

The "Operating Layer" is the safety net that turns a smart but confused robot into a reliable financial agent.

1. Problem Statement

The paper addresses the critical reliability gap in autonomous language-model (LM) agents when managing real capital in live financial markets. While existing benchmarks (e.g., SWE-bench, AgentBench) evaluate agents in synthetic or text-only environments, they fail to capture the complexities of persistent, real-world financial operations, including:

Irreversible settlement: Transactions cannot be undone.
Real economic costs: High transaction fees (2.3% per swap in this study) and slippage.
Multi-agent feedback: Agents react to the same market state and each other's trades.
Long-horizon autonomy: Agents must make thousands of sequential decisions over days without human intervention.

The authors argue that reliability in this domain is not solely a function of the base model's intelligence but is an emergent property of the operating layer—the system architecture surrounding the model that translates user intent into validated, executable actions.

2. Methodology: DX Terminal Pro

The study utilized DX Terminal Pro, a 21-day live deployment on the Base blockchain involving 3,505 user-funded vaults.

System Architecture:
- Model: Qwen/Qwen3-235B-A22B-Thinking-2507 (235B parameters, 22B active).
- Inference: Served via SGLang on a fixed compute stack to ensure consistency.
- Environment: A bounded market of 12 memecoin tokens on Uniswap V4.
- Constraints: One agent per wallet; humans could configure strategies and risk sliders but could not execute individual trades. Agents had to choose exactly one action per cycle: Buy, Sell, or Observe.
Control Loop:
- Input: Users configured vaults via structured sliders (Trading Activity, Risk, Trade Size, etc.) and natural language strategy text.
- Processing: A "harness" compiled these inputs into a prompt, including real-time market data, portfolio state, and memory.
- Validation: An off-chain policy layer checked tool calls for validity (e.g., balance checks, slippage limits, token allowlists) before submission.
- Observability: The system logged a full instruction-to-settlement trace for every invocation, linking user mandates, prompts, reasoning, tool calls, and onchain outcomes.
Scale: 7.5 million agent invocations, ~70 billion inference tokens, and ~$20M in trading volume.

3. Key Contributions

The paper makes three primary contributions:

Architecture for Real-Capital Agents: A live system design that compiles user configurations into agent context, validates actions pre-settlement, and maintains granular traces from intent to chain outcome.
Identification and Mitigation of Failure Modes: The authors identified five specific failure modes unique to persistent, real-capital operation and demonstrated how targeted "harness" (system-level) interventions reduced them.
Empirical Evidence of Operating-Layer Dominance: Data showing that reliability improvements come from system design (prompt compilation, validation, memory semantics) rather than just model upgrades.

4. Key Results & Failure Modes

The study revealed that base models often fail due to prompt interpretation errors rather than lack of reasoning capability. Through iterative pre-launch testing, the team reduced specific failure rates:

| Failure Mode | Description | Observed Shift (Pre-fix to Post-fix) | Intervention |
| :--- | :--- | :--- | : |
| Rule Fabrication | Agents invented non-existent rules (e.g., "Hierarchy Rule #2") to justify trades. | 57% $\to$ 3% of sell decisions | Removed law-like wording; clarified that prior decisions are context, not precedent. |
| Fee Paralysis | Agents refused to trade because the 2.3% fee was presented before market context, causing over-caution. | 32.5% $\to$ <10% fee-led observations | Moved fee context to follow market opportunity analysis; avoided presenting fees as the primary rule. |
| Tokenomics Misread | Agents sold tokens during "reap" events (liquidity redistribution) because they saw price crashes but missed the payout mechanics. | 42.9% $\to$ 78.0% capital deployment | Inserted structured tokenomics context (whitepaper mechanics) before price data, allowing agents to reason about payouts vs. price drops. |
| Number Hardening | Soft constraints (e.g., "Trade Size 5") were interpreted as hard numerical targets, inverting intended behavior. | Restored monotonic gradient | Replaced exact numbers with comparative language tied to market edge. |
| Cadence Trading | Agents traded based on elapsed time ("last trade was 6 ticks ago") rather than market signals. | Reduced significantly | Banned fixed cadence; filtered memory to prevent self-reinforcing rhythms. |

Production Behavior Findings:

Slider Efficacy: Structured controls (sliders) mapped user intent to behavior more reliably than free-form chat. For example, "Trade Size" sliders directly correlated with spend percentage (2% to 95% of ETH).
Herding & Cascades: Agents exhibited speculative market dynamics, including "attention cascades" where 1,500+ agents bought the same token within an hour, and sell cascades with 9.5-second inter-agent gaps.
Two-Sided Flow: Despite herding, 92.9% of 5-minute windows saw both buys and sells, driven by diverse user mandates (sliders/strategies) rather than model diversity.
Outcome Correlation: Users who provided concrete, checkable instructions (exit conditions, parameters) were 4.2x more likely to close in profit than those using vague "outperform" requests.

5. Significance & Implications

Reliability is an Operating-Layer Property: The paper demonstrates that a model can follow instructions well but still trade poorly if the prompt structure, memory semantics, or validation logic are flawed. A "harness" can improve a model's success rate from ~87% to 99.9% (as shown in internal EVM swap evaluations).
Trace-Level Observability is Critical: The ability to trace an action back to the specific line of policy, memory, or user mandate that caused it is essential for debugging. This allows for the separation of "model confusion" from "user mandate errors."
Beyond Text Benchmarks: Standard text-based benchmarks fail to capture "fee paralysis" or "fabricated rules" that only emerge under financial pressure and repeated execution.
Future Training Data: The instruction-to-settlement traces provide a rich dataset for Reinforcement Learning (RL), synthetic scenario generation, and evaluating model homogeneity across different architectures.

Conclusion:
The study concludes that for capital-managing agents, the "operating layer" (prompt compilation, typed controls, policy validation, and observability) is the primary determinant of reliability. Future progress in financial AI depends less on scaling base models and more on engineering robust, instrumented systems that can detect and correct failures before capital is deployed.