Bayesian Inference for Estimating Generation Costs in Electricity Markets

1. Problem Statement

The paper addresses the challenge of estimating generation costs (specifically marginal costs and start-up costs) from observed electricity market data, such as generation schedules. This estimation is critical for market simulation, strategic bidding, price forecasting, and regulatory monitoring.

• Context: In competitive electricity markets, participants theoretically bid at their marginal costs. However, observed schedules are the result of complex Unit Commitment (UC) problems involving physical constraints (ramp rates, capacity limits), stochastic elements (demand fluctuations, generator outages), and strategic bidding behaviors.
• The Gap: Traditional Inverse Optimization (IO) methods treat cost parameters as unknowns to be recovered from schedules but typically provide only point estimates without quantifying uncertainty. They often fail to account for the stochastic nature of the market (e.g., random outages or bidding noise), leading to model misspecification.
• Objective: To formulate cost estimation as a Bayesian inference problem, allowing for the recovery of cost parameters while rigorously quantifying the uncertainty associated with those estimates.

2. Methodology

The authors propose a framework combining a Latent Variable Model with Simulation-Based Inference (SBI).

A. Latent Variable Model

The market process is modeled as a generative process where observed schedules ($g$) and loads ($\delta$) are outcomes of underlying latent variables:

• Parameters ($\theta$): Marginal costs ($c$) and start-up costs ($s$), assigned uniform priors.
• Latent Variables ($z$):
  • Bidding costs ($\tilde{c}$): Modeled as a noisy version of marginal costs ($\tilde{c} \sim \mathcal{N}(c, \sigma^2)$) to account for strategic deviations.
  • Availability ($\alpha, \beta$): Bernoulli variables representing generator and line outages.
  • Demand ($\delta$): Modeled with seasonal and diurnal profiles plus noise.
• Forward Model: The schedule $g$ is the deterministic solution to a Security-Constrained Unit Commitment (SCUC) problem: $g = \text{UC}(\tilde{c}, s, \alpha, \beta, \delta)$.

B. Bayesian Inference Formulation

The goal is to compute the posterior distribution $p(\theta | x_{obs})$, where $x_{obs} = (g_{obs}, \delta_{obs})$.
$$p(\theta | x_{obs}) \propto p(\theta) p(x_{obs} | \theta)$$
Since the likelihood $p(x_{obs} | \theta)$ is intractable (due to the complex, non-differentiable UC optimization and latent variables), the authors avoid direct likelihood evaluation.

C. Inference Algorithms

Two methods are compared:

1. Balanced Neural Posterior Estimation (BNPE):
   • A Simulation-Based Inference (SBI) approach.
   • A neural network (specifically a Neural Spline Flow) is trained to approximate the posterior distribution $q_\phi(\theta | x)$.
   • Training: The network minimizes the expected negative log-posterior (equivalent to minimizing KL divergence) using a dataset of simulated pairs $(\theta, x)$ generated by sampling from the prior and running the forward SCUC model.
   • Advantage: Once trained, it provides fast inference with full uncertainty quantification (credible intervals).
2. Inverse Optimization (Baseline):
   • A traditional approach using the Polar Cone Method.
   • It assumes a deterministic model (ignoring stochastic bidding and outages) and iteratively refines a set of feasible cost parameters (the polar cone) such that the observed schedule is optimal.
   • Limitation: Provides only a point estimate and suffers from model misspecification when the real data is stochastic.

3. Key Contributions

• Formulation: Successfully reframes generation cost estimation as a Bayesian inference problem using a latent variable model that captures stochastic bidding, outages, and demand uncertainty.
• Algorithm Application: Introduces Balanced Neural Posterior Estimation (BNPE) to the electricity market domain, enabling amortized inference (fast repeated queries) with uncertainty quantification.
• Comprehensive Benchmarking: Compares the SBI approach against a state-of-the-art inverse optimization baseline on a realistic network (IEEE RTS-96).
• Diagnostic Framework: Implements rigorous diagnostics (posterior predictive checks, calibration curves) to validate the reliability of the learned posterior in the absence of ground truth for real-world data.

4. Experimental Results

The methods were validated on the IEEE RTS-96 test system (24 buses, 12 generators, 24-hour horizon).

• Marginal Costs ($c$):
  • BNPE: Successfully recovered marginal costs with narrow, peaked posterior distributions. The true nominal parameters consistently fell within high-density regions (68.3% and 95.5% credible intervals).
  • Inverse Optimization: Produced larger deviations from true values and often fell into low-density regions of the true posterior, failing to capture the stochastic nature of the data.
• Start-up Costs ($s$):
  • BNPE: The posterior remained diffuse (close to the prior). This is a correct inference, reflecting the physical reality that start-up costs are largely unidentifiable from a single day's schedule once a unit is committed. The model correctly identified the lack of information rather than hallucinating a specific value.
  • Inverse Optimization: Provided a point estimate but could not quantify this unidentifiability.
• Calibration & Consistency:
  • Calibration: The BNPE showed slight overconfidence (empirical coverage slightly below nominal levels), likely due to the high dimensionality (24 parameters) and complex stochastic interactions, but remained useful for decision-making.
  • Posterior Predictive Checks: Sampling parameters from the learned posterior and re-running the forward model produced schedules consistent with the observed data, confirming the model captured the inverse mapping correctly.

5. Significance and Conclusion

• Uncertainty Quantification: The primary contribution is the ability to provide credible intervals for cost estimates. This allows market analysts and regulators to understand the reliability of their estimates (e.g., distinguishing between a well-identified marginal cost and an unidentifiable start-up cost).
• Operational Efficiency: Unlike MCMC methods which are computationally prohibitive for high-dimensional UC problems, BNPE offers amortized inference. After a one-time training phase, posterior evaluation for new market data is nearly instantaneous, enabling real-time monitoring.
• Robustness: By explicitly modeling stochastic elements (outages, bidding noise), the Bayesian approach avoids the systematic errors introduced by deterministic inverse optimization methods.
• Future Directions: The authors suggest extending the model to infer physical parameters (ramp rates) jointly, utilizing flow matching for higher scalability, and incorporating multi-day data to improve the identifiability of start-up costs.

In summary, this paper demonstrates that simulation-based Bayesian inference is a superior framework for estimating electricity market parameters compared to traditional inverse optimization, offering a necessary balance between computational speed and uncertainty quantification.