A Rule-Aware Prompt Framework for Structured Numeric Reasoning in Cyber-Physical Systems

Technical Summary: A Rule-Aware Prompt Framework for Structured Numeric Reasoning in Smart Grids

Problem Statement

Modern smart grids rely on high-dimensional numeric telemetry (e.g., bus voltages, power injections, line flows) and explicit operating rules to ensure reliability. While Large Language Models (LLMs) are increasingly explored for grid operations, current deployments are predominantly text-centric, focusing on logs and alerts rather than directly reasoning over numeric measurements. When numeric grid data are naively serialized as tokens, LLMs often fail to adhere to underlying decision rules, becoming sensitive to representation choices and drifting from physical or statistical criteria. This gap is critical in safety-critical anomaly detection (AD), where decisions must be traceable to well-understood operating limits. The core challenge addressed is how to structure prompts so that LLMs can reason over numeric telemetry in a way that is faithful to power system rules, interpretable to operators, and efficient in token usage.

Methodology

The authors propose a Rule-Aware Prompt Framework that decomposes prompts into a modular architecture designed to encode domain context, measurement semantics, and decision rules. The framework consists of six distinct modules:

1. Role Instruction (R): Defines the model's persona (e.g., power system analyst) and task.
2. Domain Context (C): Describes the system, measurement types, and feature semantics to ground numeric values in physical meaning.
3. Numeric Normalization (N): Specifies the transformation of raw measurements into normalized quantities (e.g., z-scores) to reduce scale sensitivity and token redundancy.
4. Rule-Aware Reasoning (S): Encodes the decision logic (e.g., three-sigma criteria) and instructs the LLM to apply these rules explicitly.
5. Value Block (V): Presents the telemetry in a compact format (e.g., a list of normalized descriptors) aligned with the reasoning rules.
6. Output Schema (O): Constrains the response to a categorical label and a brief, machine-actionable explanation.

The framework is instantiated on the IEEE 118-bus transmission network using GPT-OSS-20B (a 20B parameter open-weight MoE model). The study evaluates five prompting and adaptation regimes:

• Zero-shot: No labeled examples.
• Few-shot: Two labeled examples (one normal, one anomaly).
• In-Context Learning (ICL): Five diverse labeled examples.
• LoRA Fine-tuning: Parameter-efficient adaptation to generate labels and rationales.
• Hybrid LLM+DL: The LLM acts as a rule-aware feature selector and explanation engine, filtering channels for a downstream deep learning (DL) detector.

A key design choice is the Value Block (V) representation. The paper compares four variants: raw values only; raw values with mean/std; raw values with mean/std/z-score; and z-score only. The proposed "z-score only" design omits raw values and statistics, relying solely on absolute z-scores aligned with the decision rule to minimize token usage while preserving reasoning information.

Key Results

The experiments, conducted on the IEEE 118-bus system with injected anomalies (15% deviation in three random sensors), yield the following findings:

• Impact of Value Block Design: In zero-shot settings, the representation of numeric data significantly affects performance. Using raw values only (Setup 1) resulted in low accuracy (36.2%). Adding statistical context (Setup 2) improved F1 to 65.3%. However, the z-score-only design (Setup 4) achieved the best zero-shot performance with 71.8% accuracy and a 77.9% F1-score, while maintaining a token count comparable to the raw-value setup (2,203 tokens vs. 2,189 tokens) and far lower than setups including raw statistics. This demonstrates that concise, rule-aligned representations outperform simply exposing more data.
• Supervision Levels: Performance improved progressively with supervision. Few-shot and ICL raised the F1-score above 80%. LoRA fine-tuning achieved 100% recall but lower precision (75.8%), resulting in an F1 of 86.2%.
• Hybrid Architecture: The Hybrid LLM+DL approach achieved the highest overall performance (94.0% accuracy, 93.6% F1-score) with 100% precision. In this setup, the LLM filtered measurement channels based on the three-sigma rule before passing them to a traditional DL detector. This configuration outperformed both the standalone DL detector (F1 88.3%) and the direct three-sigma baseline (F1 89.9%).
• Token Efficiency: The z-score-only prompt (Setup 4) offered the best balance of performance and efficiency. In contrast, ICL required over 11,000 tokens, and LoRA fine-tuning incurred higher latency due to model loading, despite similar token counts to Setup 4.

Significance and Claims

The paper claims that clarity and rule alignment in prompt design are more critical than input volume for grid monitoring tasks. The primary contributions are:

1. A modular, rule-aware prompt framework that explicitly separates role, context, normalization, rule specification, and output, enabling reusable integration across grid monitoring tasks.
2. A principled formulation where LLM decisions are based on normalized deviations (z-scores) aligned with operating criteria rather than raw measurements, ensuring decisions remain traceable to statistical or physical rules.
3. An empirical demonstration that a hybrid LLM+DL architecture can surpass traditional detectors in precision while providing interpretable rationales, effectively bridging the gap between high-dimensional telemetry and general-purpose LLMs.

The authors position the LLM not as a replacement for conventional threshold-based detectors, but as a component that integrates explicit rules, normalized evidence, and structured output constraints to enhance transparency and operator trust. Future work is proposed to incorporate richer physical rules (e.g., AC power-flow consistency) and extend the framework to distribution systems and microgrids.