⚡ electrical engineering

Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets

This paper proposes a scalable uncertainty quantification framework that integrates Monte Carlo dropout with conformal prediction to generate calibrated, finite-sample prediction intervals for aggregated prosumer flexibility, enabling demand response aggregators to reliably meet strict regulatory standards like P90 while significantly reducing overbidding risks and achieving up to 70% of perfect-information profits in ancillary service markets.

Original authors: Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg, Julian Lesmos-Vinasco

Published 2026-01-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yogesh Pipada Sunil Kumar, S. Ali Pourmousavi, Jon A. R. Liisberg, Julian Lesmos-Vinasco

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a fleet of small, independent boats (these are prosumers—people who both use and generate electricity, like homes with solar panels and batteries). Your job is to sell a promise to a giant shipping company (the energy market) that your fleet can provide extra power if a storm hits.

The problem? You can't predict exactly how much power your fleet will have available tomorrow. The sun might be cloudy, the batteries might be half-full, or the homeowners might decide to charge their electric cars instead of selling power. If you promise too much power and your fleet can't deliver, you get fined heavily. If you promise too little, you lose money.

This paper is about building a super-smart weather forecast for your fleet's power availability that doesn't just guess a single number, but gives you a safe, reliable safety zone.

Here is the breakdown of how they solved it:

1. The Problem: The "Guessing Game" is Dangerous

Traditionally, aggregators (the captains) use computer models to guess how much power they can sell.

The Old Way: They used models that gave a single "best guess." It was like saying, "We will definitely have 100 kW of power." But because of the uncertainty (clouds, human behavior), this guess was often wrong. They would promise too much, fail to deliver, and get fined.
The Risk: The energy market has a strict rule called P90. This means you must be able to deliver your promised power 90% of the time. If you fail more than 10% of the time, you are in trouble.

2. The Solution: A "Double-Check" System

The authors created a new method that combines two powerful tools to create a "safety zone" (a range of numbers) instead of a single guess.

Tool A: Monte Carlo Dropout (MCD) – The "Imagination Engine"
Imagine asking a group of experts to guess the weather, but every time they speak, they randomly forget a few facts. By asking them to guess 1,000 times with different random "forgetting," you get a wide variety of possible outcomes. This helps the computer understand what it doesn't know (uncertainty).
- The Flaw: While this creates many possibilities, the computer's "safety zone" is often too optimistic. It thinks it's safer than it actually is, leading to the old problem of over-promising.
Tool B: Conformal Prediction (CP) – The "Reality Check"
This is like a strict referee. After the "Imagination Engine" makes its guesses, the referee looks at past data to say, "Okay, to be 90% sure you don't fail, you need to shrink your safety zone by this amount." It forces the prediction to be conservative enough to meet the P90 rule.
The Hybrid (MCD + CP):
The paper combines these two. The "Imagination Engine" generates the possibilities, and the "Reality Check" tightens the safety zone until it is mathematically guaranteed to be 90% reliable.

3. The Test: The Danish Market

The researchers tested this on a real-world scenario in Denmark, using data from a sophisticated home energy system (HEMS) that controls thousands of virtual homes with solar panels and batteries.

They compared their new "Double-Check" system against the old "Best Guess" methods.

The Results:

The Old Methods: They were like gamblers. They promised huge amounts of power and made a lot of money on paper, but when the market actually checked, they failed to deliver often. They violated the P90 rule and would have been fined or banned.
The New Method: It was more cautious. It promised less power than the gamblers, but it never broke the rules.
- It successfully met the 90% reliability requirement.
- It still made money—about 70% of the "perfect world" profit (the profit you'd make if you knew the future perfectly).
- It eliminated the risk of getting fined for over-promising.

4. The Surprising Discovery: "Why Pay More?"

The study also looked at how much money the aggregator needs to pay the homeowners (prosumers) to get them to sell power.

They found that homeowners are willing to sell almost all their available power even for a very small payment.
Because the supply of power is so "inelastic" (it doesn't change much even if you pay more), the aggregator makes the most money by keeping the payment to the homeowners low.
The Catch: This suggests that under current market rules, homeowners might not be very motivated to participate actively because the extra money they get is small. The paper suggests that maybe the rules need to change to encourage more people to join in.

Summary

Think of this paper as a guide for a captain who wants to sell a shipping service without getting fined for late deliveries.

Old approach: "I promise to be there!" (Often fails, gets fined).
New approach: "I promise to be there, and I have a safety margin that guarantees I will be there 9 out of 10 times, even if the weather is weird."
Outcome: You make slightly less money than the "perfect guess" scenario, but you keep your license to operate and avoid the fines. It's a practical, safe, and profitable way to manage uncertainty.

Technical Summary: Calibrated Uncertainty Quantification for Prosumer Flexibility Aggregation

Problem Statement
Independent demand response aggregators play a pivotal role in integrating distributed energy resources (DERs) into ancillary service markets, such as the Danish manual frequency restoration reserve (mFRR) capacity market. A central challenge for these aggregators is accurately predicting the price responsiveness of prosumers (consumers who also produce energy) under conditions of significant epistemic uncertainty. This uncertainty arises from limited historical behavioral data, reliance on exogenous forecasts (weather, load, prices), and heterogeneous prosumer behaviors.

In markets like the Danish mFRR, participants face strict reliability requirements, specifically the P90 standard, which mandates that reserved capacity be available with a maximum 10% probability of deviation. Deterministic models or poorly calibrated probabilistic models are unsuitable for such environments; they often lead to overestimation of available flexibility, resulting in non-compliance, financial penalties, or market suspension. Conversely, overly conservative estimates reduce market competitiveness and profitability. The paper addresses the need for a data-driven framework that captures epistemic uncertainty while providing formal, finite-sample coverage guarantees to satisfy regulatory constraints and optimize economic outcomes.

Methodology
The study proposes a hybrid Uncertainty Quantification (UQ) framework that integrates Monte Carlo Dropout (MCD) with Conformal Prediction (CP).

Hybrid MCD–CP Framework:
- Monte Carlo Dropout (MCD): Used to capture epistemic uncertainty by retaining dropout layers during inference, approximating a deep Gaussian process. This provides a predictive distribution but often yields miscalibrated intervals.
- Conformal Prediction (CP): A distribution-free method applied to the MCD outputs to construct prediction intervals with finite-sample coverage guarantees under data exchangeability.
- Integration: The framework uses MCD to generate stochastic samples and CP to calibrate these samples into valid prediction regions. The study evaluates four multivariate conformity scores to define the prediction region $\xi$ $ξ$ :
  - MCP: Multivariate Conformal Prediction using quantile bands.
  - MMCP: Max-Mahalanobis Conformal Prediction, using the maximum Mahalanobis distance across outputs.
  - PCP: Probabilistic Conformal Prediction using the union of hyperspheres.
  - CCP: Copula Conformal Prediction, modeling joint residual dependencies.
Aggregator Bidding Model:
- The framework is embedded into a decision-dependent chance-constrained optimization problem.
- The aggregator maximizes profit by determining bid capacities and incentive prices (based on a revenue-sharing mechanism $\beta$ ) for prosumers.
- The P90 reliability constraint is enforced by using the lower bound of the calibrated prediction interval ( $l = \inf \xi$ ) as the deterministic capacity limit in the optimization, ensuring $P[y \in \xi] \geq 1 - \alpha$ .
Data Generation:
- A large-scale synthetic dataset (22,000 scenarios for 100 prosumers) was generated using a modified industry-grade Home Energy Management System (HEMS) from Watts A/S.
- Inputs included market prices, activation signals, load profiles, solar generation, and device specifications. The model predicts aggregated up-regulation flexibility.

Key Contributions
The paper claims three primary contributions relative to the state-of-the-art:

Scalable Hybrid Framework: Unlike prior studies relying on computationally expensive Bayesian Neural Networks (BNNs) restricted to short-term settings, this work applies the MCD–CP hybrid to a multi-time-step, multivariate aggregator bidding problem. It demonstrates that this approach can satisfy P90 compliance while remaining economically viable.
Systematic Comparison of Calibration: The study systematically deploys and compares standalone MCD against hybrid MCD–CP methods in a realistic multi-step optimization setting. It shows that the inherent miscalibration of standalone MCD (which systematically overestimates flexibility) is effectively mitigated by conformal calibration, yielding intervals suitable for market decision-making.
Realistic Synthetic Dataset: The authors developed a comprehensive synthetic dataset using a modified industry-grade HEMS algorithm combined with public data, enabling a realistic evaluation of both predictive reliability and economic performance under real-world market conditions.

Results

Predictive Reliability: Standalone MCD and non-conformal baselines failed to meet the P90 requirement, exhibiting near-zero joint prediction interval coverage probability (PICP) and high overbid frequencies (up to ~100% violation). In contrast, CP-based methods (MMCP, PCP, CCP) achieved the target 90% coverage. PCP was the most conservative (0.14% miscoverage), while MMCP and MCP offered a balance between reliability and interval width.
Economic Performance:
- Uncalibrated methods (e.g., sample mean) initially appeared profitable but suffered severe losses (up to -97% of perfect-information profit) when P90 penalties were applied due to frequent overbidding.
- The hybrid MCD–CP framework significantly reduced overbidding risk. The MCP and MMCP methods achieved the highest adjusted profits, retaining approximately 68–71% of the perfect-information (PI) profit while satisfying regulatory constraints.
- PCP and CCP, though more conservative, still outperformed all non-CP baselines under penalty schemes.
Incentive Sensitivity: The analysis of the revenue-sharing parameter $\beta$ revealed a weak supply elasticity (3–5%). Prosumers offered most of their available flexibility even at low incentive levels (optimal $\beta$ found to be 0.1 in 97% of scenarios), suggesting that increasing incentives yields diminishing marginal returns for the aggregator.

Significance and Claims
The paper asserts that the proposed MCD–CP framework provides a practical, computationally efficient, and market-compliant solution for aggregator flexibility forecasting. Its significance lies in bridging the gap between statistical rigor (finite-sample guarantees) and economic viability. By transforming miscalibrated probabilistic outputs into calibrated prediction intervals, the framework allows aggregators to navigate the trade-off between aggressive bidding and regulatory compliance.

The authors modestly conclude that while the framework achieves up to 70% of perfect-information profit, the current market design presents a barrier to prosumer engagement due to low marginal gains from higher incentives. They suggest that future work should explore decentralized learning (e.g., federated learning) and incentive mechanisms that explicitly account for market price uncertainty, rather than claiming immediate solutions to these broader policy issues. The study does not claim to solve the fundamental data scarcity problem but rather provides a robust method to manage the uncertainty inherent in existing data-limited environments.