⚡ electrical engineering

Learning-Augmented Power System Operations: A Unified Optimization View

This paper introduces LAPSO, a unified optimization framework that systematically integrates machine learning with traditional physics-based power system operations by jointly optimizing models across training and inference stages, providing design metrics for evaluation, and offering an open-source Python package to automate the augmentation of existing decision-making processes.

Original authors: Wangkun Xu, Zhongda Chu, Fei Teng

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wangkun Xu, Zhongda Chu, Fei Teng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the power grid as a massive, complex orchestra. For a long time, the conductor (the power system operators) has relied on a strict, rulebook-based approach (physics-based models) to keep the music playing smoothly. They know exactly how every instrument should behave based on the laws of physics.

However, the orchestra is changing. They are adding new instruments that are unpredictable and a bit chaotic—like renewable energy sources (wind and solar) that depend on the weather. The old rulebook is struggling to keep up with this new chaos, leading to potential instability or wasted money.

Enter Machine Learning (ML). Think of ML as a talented, intuitive musician who can "feel" the patterns of the weather and the grid better than the old rulebook. But here's the problem: in the past, the "intuitive musician" and the "rulebook conductor" worked in separate rooms. The musician practiced alone to be as accurate as possible, and the conductor made decisions based on the rulebook, ignoring the musician's insights until it was too late. This "siloed" approach often led to bad decisions—either too expensive or dangerously unstable.

The Solution: LAPSO (The Unified Conductor)

This paper proposes a new framework called LAPSO (Learning-Augmented Power System Operations). Think of LAPSO as a new way of conducting the orchestra where the "intuitive musician" (ML) and the "rulebook conductor" (Optimization) are trained and working together in the same room, from the very beginning.

Here is how the paper breaks this down using simple concepts:

1. The "Training" Problem: Why Accuracy Isn't Everything

Usually, when we train a machine learning model (like a weather forecaster), we just want it to be as accurate as possible.

The Paper's Insight: In the power grid, being "accurate" isn't always the most important thing.
The Analogy: Imagine a weather forecaster who predicts the wind speed perfectly. But, because of that perfect prediction, the power grid operator decides to turn off a backup generator, thinking the wind will hold. If the wind suddenly stops, the grid crashes.
LAPSO's Fix: Instead of just training the forecaster to be "right," LAPSO trains the forecaster to be useful. It asks: "Does this prediction help the conductor make the best, cheapest, and safest decision?" If a slightly less accurate prediction leads to a safer grid, LAPSO prefers that one.

2. The Two Main Ways They Work Together

The paper highlights two specific ways this new "teamwork" happens:

Stability-Constrained Optimization (SCO): The "Safety Guard"
- The Problem: The grid needs to stay stable (not wobble or crash). Traditional methods are slow to calculate this.
- The LAPSO Way: They train a machine learning model to act as a "Safety Guard." This guard learns to spot dangerous situations quickly.
- The Catch: If the guard is too complicated (like a super-complex neural network), it slows down the whole decision process. LAPSO teaches the guard to be smart but simple enough that it doesn't slow down the conductor. It balances accuracy (catching all dangers) with speed (not freezing the system).
Objective-Based Forecasting (OBF): The "Cost-Aware Forecaster"
- The Problem: Forecasters usually just try to guess the future load (how much electricity people will use).
- The LAPSO Way: This forecaster is trained specifically to minimize the cost of running the grid.
- The Analogy: Imagine a chef who doesn't just guess how many people will come to dinner, but guesses in a way that minimizes food waste and cost. If the chef knows that a slight over-estimate of guests is cheaper than running out of food, they will "bias" their guess slightly to save money. LAPSO does this for power grids: it trains the forecaster to make predictions that result in the lowest possible bill for the utility company.

3. Handling the Unknown (Uncertainty)

The paper also talks about "uncertainty."

The Analogy: Imagine you are planning a picnic. You have uncertainty about the weather (will it rain?) and uncertainty about your own plan (what if you forget the umbrella?).
LAPSO's Approach: It creates a mathematical map to track where these uncertainties come from. Is the uncertainty coming from the machine learning model itself (it's guessing)? Or is it coming from the optimization problem (the plan is flexible)? LAPSO helps the operators understand these different types of "guesses" so they can make safer decisions even when things are unclear.

4. The Toolkit (The Python Package)

Finally, the authors didn't just write a theory; they built a toolbox (a Python package called lapso).

What it does: It's like a universal adapter. If a power company already has a complex optimization model (their "rulebook"), this tool automatically plugs the new "intuitive musician" (ML) into it without breaking the existing system. It handles all the heavy math so engineers don't have to rewrite their entire code from scratch.

Summary

In short, this paper argues that we shouldn't treat Machine Learning and Power System Operations as separate tools. Instead, we should co-design them.

Old Way: Train the AI to be perfect, then hope it fits into the grid's rules.
LAPSO Way: Train the AI specifically to help the grid's rules make better, cheaper, and safer decisions, while ensuring the AI doesn't get too complicated to use.

It's about moving from a "siloed" approach (working alone) to an "integrated" approach (working as a team) to handle the chaotic, renewable-heavy power grids of the future.

Technical Summary: Learning-Augmented Power System Operations (LAPSO)

1. Problem Statement

Traditional power system operations (PSO) rely on physics-based models and sequential decision-making frameworks comprising forecasting, operation, and control. However, the increasing penetration of renewable energy and inverter-based resources (IBRs) introduces significant uncertainties and complex dynamics that challenge the economic efficiency, stability, and robustness of these traditional approaches.

Current solutions often suffer from a "siloed" design paradigm:

Isolated Tasks: Forecasting, operation, and control are designed independently. For instance, renewable forecasting is typically accuracy-driven, ignoring its cascading impact on downstream operational economics. Conversely, economic-driven operations often neglect stability constraints at the control stage.
Disconnected Learning: Machine learning (ML) models are trained in isolation from the optimization frameworks they eventually support. The objectives of downstream physics-based optimizations are not reflected in the ML training process, and the interaction between ML uncertainty and optimization parameters remains opaque.
Lack of Unified Framework: There is no general framework to systematically integrate data-driven ML with physics-based optimization, quantify the trade-offs between model accuracy and operational performance, or trace uncertainties from diverse sources (e.g., black-box models vs. unknown system parameters).

2. Methodology: The LAPSO Framework

The paper proposes Learning-Augmented Power System Operations (LAPSO), a unified framework that centers on the operation stage to systematically model the "data-physics nexus."

Core Mathematical Formulation

LAPSO redefines the basic power system operation problem ( $P_{basic}$ ) by encoding parametric ML models as inherent components.

Basic Problem ( $P_{basic}$ ): $\min_z f(z; y)$ subject to $g(z; y) \le 0$ , where $z$ are decision variables and $y$ are parameters (e.g., load, renewable profiles).
LAPSO Problem ( $P_{lapso}$ ): The framework introduces a learnable component $v(\cdot; \theta)$ that predicts parameters $\hat{y}_2$ based on inputs $x$ and decisions $z$ . The augmented problem is formulated as:
$\min_z f(z; y_1, \hat{y}_2) + f_v(z; y_1, \hat{y}_2)$
$\text{s.t. } g(z; y_1, \hat{y}_2) \le 0, \quad g_v(z; y_1, \hat{y}_2) \le \tau, \quad \hat{y}_2 = v(z, y_1, x; \theta^*)$
Here, $\theta^*$ represents the trained model parameters. Crucially, the ML model is treated as a learnable component of the optimization, meaning the training process must account for the impact of $\theta$ on the optimality, efficiency, and conservatism of $P_{lapso}$ .

Design Principles and Metrics

The framework introduces a "Design Triangle" balancing three dimensions:

Modeling Target: The physical dynamics or indices to be learned (e.g., stability indices, renewable dynamics).
ML Techniques: The algorithmic choice (e.g., linear regression, neural networks).
Class of PSO: The structure of the underlying optimization (e.g., MILP, convex, non-convex).

Key design metrics are established to quantify trade-offs:

Accuracy vs. Operational Impact: Balancing prediction accuracy against the computational burden and economic cost of the resulting optimization.
Optimality and Conservatism: Ensuring the ML-augmented problem remains within the solvable class of the original physics-based problem (e.g., preserving convexity) to guarantee global optimality.

Representative Applications

The paper demonstrates LAPSO through two specific applications:

Stability-Constrained Optimization (SCO): ML models act as stability assessors (constraints) within the operation stage. The framework advocates for "moderate-complexity" assessors that balance accuracy with the need to preserve the optimization class (e.g., using convex or mixed-integer linear approximations) to avoid excessive computational burden or false conservatism.
Objective-Based Forecasting (OBF): Instead of minimizing forecast error (e.g., MSE), the ML forecaster is trained to minimize the operational cost of the downstream optimization. This is formulated as a bi-level optimization (Stackelberg game) where the upper level optimizes forecaster parameters $\theta_h$ to minimize the loss $\ell$ derived from the lower-level operational cost.

Uncertainty Quantification

LAPSO provides a Bayesian perspective for end-to-end uncertainty tracing:

Sources: Distinguishes between ML-Uncertainty (epistemic and aleatoric) and Opt-Uncertainty (variations in optimization structures or unpredictable parameters $y_1$ ).
Propagation: Uses automatic differentiation and the implicit function theorem (via KKT conditions) to trace how uncertainties in ML parameters propagate to the final decision variables and costs.
Robustness: Proposes a "wait-and-see" robust formulation to hedge against uncertainties in optimization parameters, solved via a column-and-constraint generation (C&CG) approach.

3. Key Contributions

Unified Framework: Proposes LAPSO as a general mathematical formulation that unifies traditionally siloed tasks (forecasting, operation, control) by treating ML as an intrinsic part of the optimization problem.
Design Metrics: Introduces a systematic set of metrics to evaluate the trade-offs between ML model performance and its impact on the underlying optimization (efficiency, optimality, conservatism).
Extensibility: Demonstrates the framework's ability to close the loop in forecast-operation-control chains and handle hybrid learning-optimization tasks.
Uncertainty Tracing: Offers a systematic method to identify and quantify uncertainties from both data-driven and physics-based sources, including a novel robust training algorithm for wait-and-see uncertainties.
Open-Source Tools: Develops and releases two Python packages:
- pso: Generates scalable power system testbeds with spatiotemporal operational data.
- lapso: Automatically augments existing optimization models (based on CVXPY) with learnable components, handling the transformation of neural networks into mixed-integer linear constraints where necessary.

4. Results and Verification

The paper validates the framework through numerical simulations on IEEE 14-, 39-, 57-, 118-, and 300-bus systems.

SCO Validation: Using the IEEE 14-bus system, the study shows that different ML model complexities (e.g., convex vs. non-convex assessors) lead to distinct trade-offs between stability assessment accuracy and operational costs. A moderate-complexity assessor is shown to offer a better balance than a highly complex one that degrades economic efficiency.
OBF Validation: The framework demonstrates that training forecasters with objective-based losses (aligning with operational costs) yields different and often more economically efficient results compared to traditional accuracy-driven training.
Scalability: Scalability analysis (detailed in the Appendix) confirms the framework's applicability to larger systems (up to 300 buses).
Implementation: The lapso package successfully automates the integration of neural networks into optimization constraints, verifying the feasibility of the proposed mathematical transformations.

5. Significance and Claims

The paper claims that LAPSO represents a fundamental shift in power system decision-making from a siloed, physics-only approach to an integrated, data-physics nexus.

Paradigm Shift: It moves beyond treating ML merely as a dynamic modeling tool to viewing it as a learnable component of the optimization itself.
Transparency: By providing a unified mathematical view, LAPSO makes the relationship between ML and physics-based methods transparent, allowing for systematic evaluation of uncertainty and performance.
Practical Utility: The framework is designed to be acceptable to system operators by preserving the structure and solvability of traditional optimization problems while enhancing them with data-driven capabilities.
Future-Proofing: The framework is inherently extensible to emerging paradigms that require closed-loop integration of forecasting, operation, and control, addressing the growing complexity of decarbonized grids.

The authors emphasize that while the paper focuses on SCO and OBF as representative examples, the LAPSO framework is broadly applicable to various power system tasks, including control frameworks for microgrids and energy storage arbitrage, provided they can be formulated as optimization problems.