🤖 AI

Beyond Forecasting: The Belief-to-Trade Layer in Prediction-Market Agents

The paper introduces Raven-Agent, the first autonomous trading agent for prediction markets, which outperforms existing policies by bridging the gap between calibrated forecasting and profitable trading to achieve positive returns on archived decision data.

Original authors: Yishu Wang, Yuxuan Wang, Jiaqi Deng, Hanyang Tang

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yishu Wang, Yuxuan Wang, Jiaqi Deng, Hanyang Tang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are at a giant, high-stakes carnival game where people bet on future events: "Will it rain tomorrow?" "Will a specific politician resign by June?" "Will the price of oil hit $100?"

Most AI researchers have been trying to build the best fortune teller. They train their AI to look at the news, crunch the numbers, and say, "I am 80% sure it will rain." They measure success by how close that 80% guess is to the actual weather.

But this paper argues that being a great fortune teller isn't enough to win money. You also need to be a smart gambler.

The authors, Yishu Wang and colleagues, built a new system called Raven-Agent. They realized that while AI is getting better at guessing probabilities, it's terrible at deciding how much to bet and when to walk away.

Here is the breakdown of their idea using simple analogies:

1. The Problem: The "Confident Loser"

Imagine a fortune teller who is very confident. They say, "I'm 90% sure Team A will win!" But the betting odds are actually 90% in favor of Team A.

The AI's Mistake: If the AI bets big money on this, it might win the bet, but it loses money overall because the payout is tiny compared to the risk.
The Real Issue: Recent studies showed that even when AI gets the probability right, it still loses money because it doesn't know how to manage its wallet. It treats every "good guess" as a reason to bet the same amount, regardless of whether the bet is actually worth it.

2. The Solution: The "Raven-Agent"

The authors didn't just build a better fortune teller. They built a Trading Manager (which they call the "Belief-to-Trade Layer") that sits between the AI's guess and the actual bet.

Think of Raven-Agent as a strict casino manager who stands next to the fortune teller. The fortune teller can shout out predictions, but the manager decides if the bet happens and how much money goes on the table.

The manager has four specific rules (modules):

The Filter (Selection): The manager looks at all the predictions and asks, "Is this actually a good deal?" If the potential profit is too small compared to the risk, the manager says, "Skip it."
The Timer (Ranking): Some bets pay out tomorrow; others take six months. The manager calculates the "speed" of the profit. A small profit that happens quickly is often better than a huge profit that takes forever to arrive.
The Wallet (Sizing): This is the most important part. The manager uses a famous math formula (called the Kelly Criterion, but scaled down to be safer) to decide exactly how much to bet.
- Analogy: If the AI is only slightly sure, the manager bets a penny. If the AI is very sure and the odds are great, the manager bets a bit more. But they never bet the whole bankroll.
The Safety Net (Risk Control): The manager has a hard stop-loss rule. If the bet starts going badly, the manager cuts the loss immediately. They also make sure the AI doesn't bet too much on one single event (like not putting all your eggs in one basket). Crucially, these rules are hard-coded, meaning the AI cannot "talk its way out" of them.

3. The Experiment: The "Time-Travel Replay"

To test this, the authors didn't just let the AI gamble with real money immediately. They used a "Time-Travel Replay."

They took a huge archive of past betting markets (from a platform called Polymarket).
They took the exact same guesses that other AIs had made in the past.
They ran those same guesses through their new "Raven-Agent" manager.
They compared the results against other strategies (like "bet $10 on everything" or "bet more if you are more confident").

4. The Results: The Only Winner

The results were clear:

The Old Ways: Every other strategy lost money. Some lost a little, some lost a lot (one lost 55% of its money!).
Raven-Agent: It was the only strategy that made a profit. It turned a losing situation into a winning one just by changing how the bets were placed, not by changing the guesses themselves.

The Takeaway

The paper claims that in prediction markets, how you manage your money is just as important as how well you predict the future.

You can have the smartest AI in the world, but if it doesn't have a strict, logical manager to tell it when to bet, how much to bet, and when to stop, it will lose money. Raven-Agent proves that adding a disciplined, mathematical "trading layer" on top of an AI can turn a losing game into a winning one.

In short: Don't just build a better crystal ball; build a better wallet manager.

Technical Summary: Beyond Forecasting: The Belief-to-Trade Layer in Prediction-Market Agents

1. Problem Statement

While forecasting future events has emerged as a critical testbed for general-purpose AI, a significant gap exists between calibrated probability scores and actual trading profitability. Existing benchmarks (e.g., ForecastBench, Prediction Arena) demonstrate that strong probabilistic forecasts do not necessarily translate into profitable trades.

The core issue identified is that current evaluation frameworks often treat the trading layer—the mechanism converting forecasts into actions—as a fixed protocol (e.g., fixed bet sizes, simple confidence thresholds) or omit it entirely. Consequently, critical downstream components such as position sizing, portfolio exposure, and risk control are either absent or merely expressed as prompt instructions within the language model. The authors argue that these components should be treated as explicit, designable, and modular objects rather than hard-coded rules or prompt-dependent heuristics.

2. Methodology: Raven-Agent

The paper proposes Raven-Agent, an autonomous trading agent designed to isolate and optimize the "belief-to-trade" layer. The system is built around a deterministic, modular architecture that sits downstream of a swappable forecasting module.

Architecture Overview

Raven-Agent consists of four primary modules:

Information Collection: Scans candidate markets (via Polymarket Gamma API), retrieves price and liquidity data, and fetches recent news evidence. This module produces a timestamped "pulse" that serves as a contract between data sources and downstream logic, allowing for modular updates to data feeds.
Probability Analysis: A swappable forecasting module that outputs the probability ( $p_i$ ) for a specific contract side. In the reported experiments, this module is held fixed, reusing archived probabilities to ensure that performance differences are attributable solely to the trading layer.
Ranking and Sizing:
- Selection: Candidates are ranked by a time-normalized return rate ( $\rho_i$ ) rather than raw probability edge ( $p_i - q_i$ ). This accounts for the duration capital is locked until resolution:
  $\rho_i = \left(\frac{p_i}{q_i} - 1\right) \cdot \frac{30}{\max(T_i, 1)}$
  where $T_i$ is days to resolution. The top $K=4$ candidates are retained.
- Sizing: Stakes are determined using a quarter-Kelly criterion applied to retained positions:
  $s_i = \frac{1}{4} \cdot \max\left(0, \frac{p_i - q_i}{1 - q_i}\right) \cdot W$
  where $W$ is the bankroll. This scaling accounts for input noise in the probability estimates.
Execution and Risk Control: This module enforces deterministic constraints outside the language model prompt to prevent "confidently wrong" failures. It checks orders against five hard constraints:
- Per-trade notional cap.
- Aggregate exposure cap.
- Per-event cap (for correlated contracts).
- Position-level stop-loss (closes positions if mark-to-market loss exceeds a fraction of cost basis).
- Portfolio-level drawdown halt (freezes new openings if equity drawdown exceeds a threshold).

Experimental Protocol

The authors introduce a controlled replay protocol using archived decision sets from a live Polymarket deployment. This protocol reuses archived forecasts, market prices, and execution records. By holding the forecaster constant and varying only the trading policy, the study isolates the impact of the trading layer.

3. Key Contributions

The paper outlines three primary contributions:

Replay Protocol: A methodology to reuse archived forecasts to isolate the trading layer from the forecaster, enabling fair comparison of trading policies.
Raven-Agent: An autonomous agent architecture that makes the trading layer an explicit, modular, and composable component, separating selection, sizing, and risk control from the forecasting model.
Empirical Validation: Evidence that an explicit trading layer improves return on stake and risk-adjusted returns compared to heuristic policies, even when the underlying forecaster is fixed.

4. Results

The agent was evaluated against five policies on a dataset of 59 open decisions across 65 pulse runs:

Forecast-only: Fixed $10 stake for every positive-edge forecast.
Edge-proportional: Stake sized proportionally to edge ( $p_i - q_i$ ) without selection filtering.
Edge-filter: Fixed $10 stake only for forecasts with edge > 5%.
Raven-Agent (fixed): Full risk layer with fixed $10 stake.
Raven-Agent (full): Full risk layer with quarter-Kelly sizing.

Key Findings:

Raven-Agent (full) was the only policy to achieve a positive Return on Investment (ROI) of +15.9% and a positive stake-weighted Sharpe ratio (+0.42).
Edge-proportional performed the worst (-55.5% ROI), demonstrating that sizing capital heavily on "confidently wrong" high-edge predictions without selection filtering amplifies losses.
Raven-Agent (fixed) (-4.7% ROI) outperformed all other fixed-stake policies, indicating that selection and risk filtering alone provide value.
The Selected Brier score improved as policies became more aggressive in filtering (from 0.329 for the full archive to 0.258 for Raven-Agent), confirming that the trading layer successfully filters out poorly calibrated forecasts.

In a live deployment (20 predictions), the agent accumulated $53.92 in profit.

5. Significance and Claims

The paper claims that the gap between forecasting accuracy and trading profitability is largely due to the lack of a robust, explicit trading layer. The authors argue that:

Selection and Risk Filtering are Critical: Simply sizing bets based on edge is insufficient; filtering out value-destroying forecasts is a necessary complement to informed sizing.
Deterministic Constraints are Superior to Prompt Guidance: Relying on prompt instructions for risk management is unreliable. Enforcing hard constraints (stop-loss, drawdown limits) outside the language model prompt prevents catastrophic failures and provides auditable rejection reasons.
Modularity Enables Progress: By decoupling the forecaster from the trading layer, improvements in calibration (forecasting) and execution (trading) can be evaluated and developed independently.

The authors maintain a modest tone, noting that the sample size is modest and the forecaster was held fixed. They acknowledge that while the results isolate the trading layer's contribution, a stronger forecaster could be plugged into the same infrastructure to further compound benefits. The work does not claim to solve all challenges in AI trading but establishes a foundational framework for the "belief-to-trade" layer.