Technical Summary: Probabilistic Prediction Markets with Intermittent Contributions

Problem Statement

The paper addresses the challenge of improving forecast accuracy in domains like renewable energy, where data availability is increasing but collaboration is hindered by data ownership, competitive interests, and privacy concerns. While existing cooperative game-theoretical frameworks and privacy-preserving distributed learning methods exist, they often assume rational, truthful, and continuous participation from agents. In practice, stakeholders may be unwilling to share data, or their participation may be intermittent.

The authors propose a prediction market framework where independent agents (sellers) trade probabilistic forecasts for uncertain future events in exchange for rewards. The specific technical challenges addressed are:

1. Real-time implementation: The need for online learning without a distinct training phase.
2. Historical performance: Accounting for the long-term informational value of agents, not just instantaneous accuracy.
3. Intermittent participation: Handling scenarios where agents enter or exit the market at will, resulting in missing forecast submissions (a missing data problem).
4. Fair reward allocation: Designing a pay-off mechanism that balances in-sample consistency (historical contribution) with out-of-sample accuracy (genuine forecast quality) while satisfying economic properties like budget balance and truthfulness.

Methodology

The proposed market design involves a central Market Operator interacting with Clients (forecast buyers) and Sellers (forecasters). The system operates in discrete time steps $t$, handling nonparametric probabilistic forecasts represented by quantiles.

1. Forecast Combination via Adaptive Robust Regression

To combine forecasts from multiple sellers, the authors employ a linear regression framework updated via Online Gradient Descent (OGD).

• Objective Function: The model minimizes the quantile loss (pinball loss) to optimize weights for specific nominal quantile levels $\tau$.
• Handling Missing Data: To accommodate intermittent participation, the authors extend a Robust Linear Regression (RQR) model. They introduce a binary availability vector $\alpha_t$ and a linear correction matrix $D_t$. When a seller's forecast is missing, the model dynamically adjusts the combination weights using the correction matrix to compensate for the missing information, effectively extracting additional information from remaining available forecasts.
• Online Update: The weights $w_t$ and correction matrix $D_t$ are updated iteratively using sub-gradients of the quantile loss. The method supports mini-batch updates to handle multi-step forecasting horizons efficiently.

2. Pay-off Allocation Mechanism

The total reward $U_t$ provided by the client is split into two components for each seller $i$ and quantile $\tau$:
$$r^{(\tau)}_{i,t} = U^{(\tau)}_t [\delta r^{is}_{i,t} + (1-\delta) r^{oos}_{i,t}]$$
where $\delta$ is a weighting parameter.

• In-Sample Allocation ($r^{is}$): Designed to reward consistency and cumulative informational value.
  • Utilizes an online variant of the Shapley value ($\phi^c_{i,t}$), which is updated recursively over time ($\phi^c_{i,t} = \lambda \phi^c_{i,t-1} + (1-\lambda)\phi^s_{i,t}$).
  • The Shapley value is calculated based on the marginal contribution of a seller to the aggregated forecast using the corrected weights.
  • Missing sellers receive zero in-sample reward.
• Out-of-Sample Allocation ($r^{oos}$): Designed to reward instantaneous forecast accuracy.
  • Based on a scoring function derived from the quantile loss.
  • Sellers are rewarded proportionally to their accuracy relative to other present sellers.
  • Missing sellers receive zero out-of-sample reward.

3. Economic Properties

The authors formally define and prove that the allocation mechanism satisfies:

• Budget Balance: The sum of distributed rewards equals the total utility generated.
• Symmetry: Identical forecasts yield identical rewards.
• Zero-Element: Non-participating sellers receive no reward.
• Individual Rationality: Rewards are non-negative.
• Truthfulness (Incentive Compatibility): Sellers maximize expected rewards by reporting true forecasts, as altering forecasts increases loss and reduces both in-sample (via Shapley value) and out-of-sample rewards.

Results

The methodology was evaluated using synthetic data and a real-world wind power forecasting case study.

Synthetic Case Studies

• Time-Invariant Scenario: Both standard Quantile Regression (QR) and Robust QR (RQR) converged to optimal weights. RQR successfully handled random missingness (5% rate) with stable convergence.
• Time-Varying Scenario: The methods successfully adapted to evolving weight distributions in a dynamic environment, tracking periodic changes in the optimal combination.
• Varying Missingness: RQR demonstrated lower variance and comparable or superior bias compared to benchmark imputation strategies (mean and last-value imputation) across missingness rates from 5% to 90%.

Real-World Case Study (Wind Energy)

• Setup: Day-ahead probabilistic forecasting for a Belgian offshore wind farm using nine input models (combinations of three NWP sources: ECMWF, NOAA, DWD, and three model types: XGBoost, QRF, MLP).
• Performance:
  • The combined forecast (QR) outperformed all individual models, confirming the value of heterogeneous information sources.
  • RQR Robustness: Even with 10–20% missing data, RQR achieved quantile loss values very close to the "oracle" QR benchmark (full information) and significantly better than the best individual model.
  • Adaptability: The moving average loss of RQR remained consistently lower than the single best expert, particularly during periods where individual experts underperformed.
• Reward Dynamics: The reward distribution highlighted that models with distinct predictive value (e.g., ECMWF-MLP, ECMWF-XGB) accumulated the highest sustained rewards. The system successfully tracked time-varying performance, such as the rising contribution of DWD-MLP in the latter half of the test period.

Significance and Claims

The paper claims to introduce a prediction market design that effectively bridges the gap between theoretical cooperative frameworks and practical, real-world constraints. Its primary significance lies in:

1. Handling Intermittency: It is the first extension of robust linear regression for forecast combination to an online learning setting, allowing agents to join and exit freely without degrading the aggregate forecast.
2. Dual-Objective Rewarding: It proposes a novel pay-off mechanism that simultaneously rewards historical consistency (via time-varying Shapley values) and instantaneous accuracy (via out-of-sample scoring), addressing a limitation in existing literature that often focuses solely on in-sample performance.
3. Economic Viability: The mechanism is proven to satisfy key economic properties, ensuring fairness and incentivizing truthful reporting.

The authors conclude that while the proposed market demonstrates effectiveness in renewable energy forecasting, future work could explore fully dynamic participant sets and alternative allocation mechanisms to further reduce computational complexity and enhance scalability.