🤖 machine learning

Revisiting data-driven dynamic security assessment with a tabular foundation model

This paper proposes a tabular foundation model for data-driven dynamic security assessment that leverages in-context learning to evaluate multiple power system contingencies with high accuracy using minimal labeled data, thereby overcoming the scalability and generalization limitations of existing machine learning approaches.

Original authors: Olayiwola Arowolo, Maosheng Yang, Jochen Cremer

Published 2026-07-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Olayiwola Arowolo, Maosheng Yang, Jochen Cremer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the power grid as a giant, invisible nervous system stretching across a continent, carrying electricity like lightning through veins of copper. Just like a human body, this system needs to stay stable; if a sudden shock hits—like a tree falling on a power line or a storm knocking out a substation—the whole thing could go into a seizure, leading to a massive blackout. To prevent this, grid operators play a high-stakes game of "what if." They constantly ask: "If this specific line breaks, will the system survive?"

Traditionally, answering this question is like trying to predict the weather by running a supercomputer simulation for every single possible storm scenario. It's accurate, but it takes so long that by the time you get the answer, the storm has already passed. So, scientists started using "machine learning," which is like teaching a computer to recognize patterns. Instead of running a slow simulation every time, the computer learns from past examples to instantly guess if a situation is safe or dangerous. However, the old way of teaching these computers was clunky: you had to train a separate "student" for every single type of accident, and if a brand-new, weird accident happened, the students didn't know what to do. This new paper explores a smarter way to teach these computers, using a "foundation model" that acts like a super-genius tutor who can learn from just a few examples and handle any emergency without needing to go back to school.

The Problem: Too Many Students, Not Enough Time

In the world of power grids, "Dynamic Security Assessment" (DSA) is the process of checking if the grid can handle a sudden shock. The old method was a bit like a school where every teacher only knew how to handle one specific type of disaster. If a fire started, you called the fire teacher; if a flood came, you called the flood teacher. But in a real power grid, there are hundreds of possible things that could go wrong. Training a separate computer model for every single possibility takes a massive amount of time and data.

Worse, if a brand-new type of accident happens—something the teachers have never seen before—they freeze up. They can't guess because they were only trained on the specific disasters they memorized. This is a big problem because power grids are changing fast, and we can't afford to wait for a computer to learn a new lesson every time a new problem appears.

The Solution: The "Super Tutor" (Tabular Foundation Model)

The authors of this paper propose a new approach using something called a Tabular Foundation Model (TFM). Think of this not as a student who memorizes answers, but as a super-genius tutor who has read millions of books about how systems behave. This tutor doesn't need to be retrained for every new question. Instead, it uses a trick called "in-context learning."

Imagine you are taking a test. In the old way, you had to study for weeks before the test. With this new tutor, you can just hand it a few example problems right before the test starts. The tutor looks at those examples, figures out the pattern, and immediately solves the new problems. It doesn't need to "relearn" anything; it just uses its existing brainpower to adapt instantly.

What They Did and Found

The researchers tested this idea on a standard computer model of a power grid called the IEEE 68-bus system. They wanted to see if this "Super Tutor" could predict grid safety better than the old methods, especially when it didn't have much data to work with.

1. Less Data, Better Results
The most surprising finding is how little data the tutor needs. In the past, experts thought you needed thousands of examples to train a model for each disaster. The researchers found that this new TFM could achieve an average accuracy score (called a Macro F1 score) of about 90% using only 120 labeled examples per disaster. That is roughly two orders of magnitude (or 100 times) fewer examples than usually thought necessary. Even more impressively, it did this without any retraining or tweaking of settings. It just worked.

2. One Model to Rule Them All
Instead of training 22 different models for 22 different disasters, the researchers used just one single TFM to handle all of them at once. This single model outperformed the best existing methods that tried to do the same thing. It was particularly good at spotting dangerous situations (insecure cases) even when those dangerous cases were very rare in the training data—a common problem in the real world where blackouts are rare, but we need to catch them every time.

3. Guessing the Unseen
The real magic happened when they tested the model on a disaster it had never seen before.

Zero-Shot (No new data): If they gave the model no new examples of the new disaster, it used a special "map" called Electrical Distance Coordinates (EDC). This map describes where the disaster happened in the grid. Using this map, the model could guess reasonably well, but it wasn't perfect.
Few-Shot (Just a few examples): The researchers then tried giving the model just 10 new examples of the unseen disaster. With these tiny hints, the model's performance jumped up to match the "best possible" theoretical model (an "oracle" that knows everything about the new disaster). This is huge because, in the real world, you can't wait to collect thousands of examples of a new disaster before acting; you need to act fast with very little information.

The Catch and the Future

The paper is careful to note that while this is a simulation on a computer model, the results are very promising. The "Super Tutor" isn't a magic wand that solves everything instantly. The researchers point out that for very large power grids, the model might get too heavy to run quickly, and it still needs human operators to double-check the answers before making life-or-death decisions.

However, the study suggests a major shift in how we might protect our power grids. Instead of building a library of thousands of specific models, we might soon rely on one flexible, smart model that can learn from a handful of examples and handle almost any emergency. It turns the difficult task of predicting grid safety from a slow, data-hungry process into something fast, efficient, and surprisingly adaptable.

Technical Summary: Revisiting Data-Driven Dynamic Security Assessment with a Tabular Foundation Model

1. Problem Statement

Data-driven pre-fault Dynamic Security Assessment (DSA) aims to rapidly evaluate the stability of power systems under credible contingencies using machine learning (ML), bypassing the computational intensity of traditional time-domain simulations (TDS). However, existing data-driven approaches face two critical limitations:

Model Proliferation and Cost: Conventional workflows require training, tuning, and maintaining a separate classifier for every single contingency in a potentially long list. This creates a significant computational burden for generating the large labeled databases required for each model.
Poor Generalization: Trained models often fail to generalize to unseen contingencies (new fault locations or topologies) or changing system conditions. Adapting to a new contingency typically requires generating new labeled data via TDS and retraining a new classifier, a process that is slow and resource-intensive.

The core challenge is how to achieve high-accuracy DSA with a limited computational budget for data generation while ensuring robust generalization to unforeseen scenarios without constant model retraining.

2. Methodology

The authors propose a novel workflow utilizing a Tabular Foundation Model (TFM), specifically adopting TabPFN, to address these limitations.

A. Tabular Foundation Model (TFM)

Unlike conventional classifiers that learn parameters via gradient descent on specific training data, the TFM is a transformer-based model pretrained on 100 million synthetic datasets. It operates via In-Context Learning (ICL):

No Retraining: The model requires no parameter updates or hyperparameter tuning for new tasks.
Inference Mechanism: Given a set of labeled training samples (context) and unlabeled queries, the TFM predicts labels in a single forward pass by "attending" to the context. It approximates the Bayesian posterior predictive distribution $P(Y_{test}|X_{test}, X_{train}, Y_{train})$ .
Sample Efficiency: The model is designed to perform accurately with very small training datasets.

B. Proposed Workflow

The workflow integrates the TFM into pre-fault DSA:

Offline Phase: A training database is generated via TDS for a set of credible contingencies.
Online Phase:
- Single/Multi-Contingency Assessment: A single TFM can assess multiple contingencies simultaneously. For a specific contingency, its labeled training data serves as the "context" for the TFM to predict stability for new operating conditions (OCs).
- Unseen Contingencies: To handle new contingencies not in the training set, the authors propose encoding the contingency scenario into the input features.

C. Scenario Encoding: EDC and mod-EDC

To enable generalization to unseen contingencies, the authors investigate continuous representations of the fault scenario:

Electrical Distance Coordinates (EDC): A continuous vector representing the electrical distance from the faulted bus to a set of reference nodes. This replaces discrete fault indices, allowing the model to interpolate between similar fault locations.
Modified EDC (mod-EDC): Recognizing that EDC alone cannot distinguish faults at the same location cleared by different lines, the authors propose a modification that incorporates post-fault topology.
- $Q_k = EDC^{pre}_k - EDC^{post}_k$
- The final encoding $M_k$ concatenates the pre-fault EDC and the differential coordinate $Q_k$ . This captures the change in synchronizing power caused by the clearing action, serving as a label-free proxy for the severity of the topology change.

3. Key Contributions

First Application of TFM to Pre-fault DSA: The paper demonstrates that a TFM can achieve accurate DSA predictions with a training database two orders of magnitude smaller than conventionally assumed (e.g., 120 samples vs. 12,000), without any model retraining or hyperparameter optimization.
Unified Multi-Contingency Classifier: The authors propose a workflow where a single TFM assesses multiple contingencies, eliminating the need for one classifier per contingency. This approach outperforms state-of-the-art multi-label learning baselines (MDSA-S), particularly in scenarios with severe class imbalance.
Characterization of Generalization: The study characterizes the conditions under which EDC encoding enables generalization to unseen contingencies. It demonstrates that while EDC is effective for zero-shot learning, the proposed mod-EDC encoding significantly improves generalization when a few labeled samples (e.g., 10) of the new contingency are available for few-shot adaptation.

4. Experimental Results

The approach was evaluated on the IEEE 68-bus system with 22 credible contingencies and 120,000 generated samples per contingency.

Sample Efficiency: With only 120 labeled samples per contingency, the TFM achieved an average Macro F1 score of ~90%. This outperformed baseline models (XGBoost, MLP, DT, SVM-RBF) which required significantly more data to reach similar performance.
Specificity: The TFM correctly classified up to 80% of insecure cases (high specificity) with only 120 samples, a critical metric for DSA.
Multi-Contingency Performance: In a pooled setting (22 contingencies), the TFM outperformed the MDSA-S baseline in Balanced Accuracy (93.86% vs. 83.43%) and Macro F1 (94.59% vs. 85.99%). The TFM showed superior robustness in "hard" contingencies with rare insecure cases, where MDSA-S often defaulted to predicting the majority class.
Generalization to Unseen Contingencies:
- Zero-Shot: EDC encoding outperformed one-hot encoding for unseen contingencies.
- Few-Shot: Using just 10 additional labeled samples of a new contingency with mod-EDC encoding allowed the TFM to match the performance of an "oracle" transfer learning model (JDA-best) that theoretically requires full labeled data from the target domain.
- Comparison: The TFM+10 approach (few-shot) achieved a Balanced Accuracy of 0.88, matching the theoretical upper bound of Joint Domain Adaptation (JDA) methods, without requiring model fine-tuning.

5. Significance and Claims

The paper claims that this initial study paves the way for developing and deploying foundation models for power system operations. The significance lies in:

Reducing Computational Burden: By requiring drastically fewer labeled samples, the approach significantly reduces the computational cost of offline database generation via TDS.
Operational Flexibility: The ability to generalize to unseen contingencies with minimal data (or even zero-shot with EDC) addresses a key barrier to the practical adoption of ML-based DSA.
Scalability: The workflow removes the combinatorial explosion of training separate models for every contingency, offering a scalable solution for large power systems.

The authors note limitations, including the current input size constraints of the TFM (100k samples, 2k features) and the fact that, like all ML-based DSA, the model does not guarantee perfect stability prediction and requires operator validation. However, the results suggest a viable path toward more efficient and robust dynamic security assessment.