💰 quantitative finance

A Scalable Bilevel Framework for Renewable Energy Scheduling

This paper proposes a scalable bilevel framework that utilizes strong duality and McCormick envelopes to relax complex renewable energy scheduling problems into linear programs, enabling efficient and accurate day-ahead contract adjustments that significantly reduce system costs, particularly under high renewable penetration levels.

Original authors: Dongwei Zhao, Vladimir Dvorkin, Stefanos Delikaraoglou, Alberto J. Lamadrid L., Audun Botterud

Published 2026-08-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dongwei Zhao, Vladimir Dvorkin, Stefanos Delikaraoglou, Alberto J. Lamadrid L., Audun Botterud

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the electric grid as a massive, high-stakes game of musical chairs played by thousands of players. The goal is to keep the lights on for everyone, but the music is unpredictable. In this game, the "chairs" are the electricity we need, and the "players" are power plants. Some players, like coal or gas plants, are reliable; they show up exactly when told and sit down on command. But the newest, most exciting players are renewable energy sources like wind and solar. They are fantastic, but they are also a bit chaotic. The wind might blow harder than expected, or the sun might hide behind a cloud. This unpredictability is the central puzzle of modern energy: how do you plan a schedule for tomorrow when you aren't 100% sure what the weather will do?

To manage this, electricity markets usually play in two rounds. First, there is the "Day-Ahead" market, where operators try to guess the weather and schedule power plants for the next day. Then, there is the "Real-Time" market, which acts like a frantic emergency room. If the wind didn't blow as predicted, or if a cloud blocked the sun, the system has to quickly pay other plants to speed up or slow down to fix the imbalance. The problem is that the first round often ignores the cost of the second round. It's like packing for a trip based on a sunny forecast, only to realize you forgot an umbrella and have to buy an expensive one at the airport later. This paper tackles the math behind making that first packing decision smarter, so the expensive airport umbrella isn't needed.

The authors, a team from MIT and Lehigh University, are working on a way to make these two rounds of the electricity market talk to each other better without breaking the bank. They propose a new mathematical framework that acts like a "crystal ball" for the day-ahead market. Instead of just guessing the average amount of wind or solar power, their method asks: "If we schedule a little less (or a little more) of this renewable energy today, how much money will we save (or lose) when we have to fix the mess in real-time?"

However, there's a catch. Calculating this "what-if" scenario is incredibly hard. It's a "bilevel" problem, which is a fancy way of saying it's a game within a game. The top level (the day-ahead planner) tries to pick the best schedule, but the bottom level (the real-time fixer) is constantly reacting to that schedule. Solving this mathematically is like trying to solve a Rubik's cube while someone else is spinning the table it's sitting on. The authors found that the old way of solving this puzzle—using a method called Mixed-Integer Linear Programming—was too slow. When they tried to apply it to a real-world system as big as New York's power grid (the NYISO system with 1,814 buses), the computer would get stuck and take hours, or even days, just to think about it. That's too slow for a power grid that needs answers in minutes.

So, the team invented a clever shortcut. They used a mathematical trick involving "strong duality" and something called "McCormick envelopes." If you imagine the complex, twisted math problem as a tangled ball of yarn, their method is like gently pulling the yarn straight so it becomes a neat, simple line. This transformation turns the impossible-to-solve "game within a game" into a standard, easy-to-solve linear problem. It's like realizing that while you can't predict the exact path of every single raindrop, you can perfectly predict the total amount of water in a bucket if you know the size of the bucket and the rain rate.

When they tested this new, streamlined method, the results were impressive. On a smaller test system (the IEEE 118-bus system), their method produced results that were almost identical to the "perfect" theoretical solution, with a tiny gap of just 0.7% in total system cost. More importantly, on the massive NYISO system, their method solved the problem in just a few minutes. In contrast, the old method failed to find a solution even after two hours of running.

The study also revealed that this smart scheduling trick gets even more valuable as we add more renewable energy to the grid. When they simulated a scenario where wind and solar made up 70% of the power mix, their method reduced the total system cost by more than 15% compared to the old, "myopic" (short-sighted) way of just guessing the average. This suggests that as the world moves toward a greener future with more wind and solar, having a scheduler that understands the cost of uncertainty isn't just a nice-to-have; it's a massive money-saver. The authors conclude that their framework offers a practical, scalable tool for market operators to keep the lights on efficiently, even when the weather is playing tricks.

Technical Summary: A Scalable Bilevel Framework for Renewable Energy Scheduling

Problem Statement
The integration of Variable Renewable Energy Sources (VRES) into electricity markets presents a challenge for existing market designs, which were originally structured for dispatchable, fossil-fueled generators. Current short-term markets operate via a sequential two-settlement process: a Day-Ahead (DA) market and a Real-Time (RT) market. In the DA market, VRES generation is typically scheduled based on a single-valued point forecast (e.g., conditional expectation), ignoring the full distribution of uncertainty. This "myopic" approach fails to account for the potential re-dispatch costs incurred in the RT market to balance generation and demand imbalances.

While previous works have proposed co-optimizing DA and RT operations to minimize total expected costs, these approaches are often incompatible with the existing sequential market-clearing structure. Alternative bilevel optimization frameworks have been proposed to adjust DA VRES quantities to internalize RT imbalance costs while maintaining the sequential structure. However, solving these bilevel problems via standard Mixed-Integer Linear Programming (MILP) reformulations (using Karush-Kuhn-Tucker conditions) has proven computationally intractable for large-scale systems, such as the New York Independent System Operator (NYISO), limiting their practical application.

Methodology
The authors propose a scalable bilevel framework to determine the optimal day-ahead VRES quantity offers that minimize total expected system costs (DA generation costs plus expected RT re-dispatch costs).

Bilevel Formulation (Problem BiD):
- Upper Level: The system operator determines the DA VRES quantity offers ( $W$ ).
- Lower Level: Given the upper-level offers, the DA market clears to determine the generation schedule ( $\Phi^{DA}$ ). Subsequently, for various real-time scenarios ( $\omega$ ), the RT market clears to minimize re-dispatch costs ( $\Phi^{RT}_\omega$ ), constrained by the DA schedule.
- The objective is to minimize the sum of DA costs and the expected RT costs.
Scalable Solution Technique (Problem BiD-McCormick):
To overcome the computational limitations of the standard MILP approach, the authors propose a relaxation technique that transforms the bilevel problem into a single-level Linear Programming (LP) problem:
- Strong Duality: Instead of using KKT conditions which introduce binary variables (leading to MILP), the authors utilize the strong duality theorem of Linear Programming. The lower-level DA problem is replaced by its primal feasibility, dual feasibility, stationarity conditions, and the strong duality constraint (equating primal and dual objectives).
- McCormick Envelope Relaxation: The strong duality constraint introduces a bilinear term ( $\lambda^W_k \cdot W_k$ ) involving the dual variable of the VRES capacity constraint and the VRES offer quantity. The authors relax this non-convex term using McCormick envelopes, which provide a convex relaxation based on known bounds for the variables.
- This transformation results in a single-level LP problem (BiD-McCormick) that can be solved efficiently without integer variables.

Key Contributions

Algorithmic Scalability: The paper introduces a technique based on strong duality and McCormick envelopes that relaxes the bilevel VRES quantity adjustment problem into a linear program. This allows for the solution of large-scale systems (e.g., 1814-bus NYISO) in minutes, whereas the conventional KKT-based MILP approach fails to converge within reasonable timeframes (e.g., >2 hours) for such systems.
Market Compatibility: The framework maintains compatibility with the existing sequential market-clearing structure, unlike co-optimization models that require simultaneous DA and RT clearing.
Quantitative Validation: The authors validate the method on both the IEEE 118-bus system and the NYISO system, demonstrating that the relaxation achieves high accuracy and significant scalability.

Results
Numerical studies were conducted on the IEEE 118-bus system and the 1814-bus NYISO system under varying VRES penetration levels (up to 70%) and transmission capacities.

Accuracy: On the IEEE 118-bus system, the proposed BiD-McCormick approach achieved a system cost within a 0.7% gap of the "Stochastic Dispatch" (StD) benchmark, which represents the theoretical lower bound of total expected costs.
Scalability: For the NYISO system, the BiD-McCormick method solved instances with up to 50 uncertainty scenarios in minutes (ranging from ~3 to 10 minutes depending on scenario count). In contrast, the KKT-based MILP approach failed to find a solution within 2 hours.
Economic Benefit: The benefits of the bilevel adjustment are more pronounced at high VRES penetration levels. On the IEEE 118-bus system with 70% penetration, the proposed method reduced system costs by over 15% compared to the "Myopic Dispatch" (MyD) strategy (where VRES offers are set at the forecast mean). In the NYISO case with high penetration, the cost reduction compared to MyD was approximately 2.5% in the specific test case, though the authors note this gap may widen with the inclusion of ramping constraints and unit commitment in future work.

Significance and Claims
The paper claims that the proposed framework serves as a practical decision-support tool for market operators. It enables the efficient computation of optimal VRES quantity adjustments that account for real-time uncertainty without disrupting the sequential market structure. The authors emphasize that while the method is a relaxation, it yields solutions with high accuracy relative to the stochastic benchmark while offering the computational tractability required for real-world, large-scale power systems. The work highlights that the economic value of such uncertainty-aware scheduling increases significantly as the penetration of renewable energy grows. Future work is identified as including ramping constraints, unit commitment decisions, and further tightening of the McCormick envelope.

Technical Summary: A Scalable Bilevel Framework for Renewable Energy Scheduling

More like this