On the Challenges of Holistic Intrusion Detection in ICS

1. Problem Statement

Industrial Control Systems (ICS) are increasingly connected to the Internet, exposing safety-critical infrastructure to sophisticated adversaries. Traditional Intrusion Detection Systems (IIDS) for ICS typically focus on isolated dimensions, such as network traffic (e.g., packet timings) or physical process states (e.g., sensor readings).

• The Gap: Real-world attacks often manifest simultaneously in both the network and the physical process. Relying on isolated detectors requires deploying multiple parallel systems, which complicates operation, parameter tuning, and alert correlation.
• The Goal: The authors aim to develop a Holistic IIDS that monitors both network behavior and physical process states within a single, unified model to capture the complete benign behavior of an ICS.

2. Methodology and Research Approach

The authors pursued a research path to build a single holistic model rather than an ensemble of specialized detectors. They explored three primary technical avenues to achieve this, encountering significant hurdles in each:

• Process Mining for Discretization: They utilized process mining techniques to model both network traffic and physical states. Since process mining requires discrete labels (bins) but ICS data (e.g., temperature, fill levels) is continuous, they investigated various discretization methods (statistical methods, unsupervised learning like clustering) to map continuous data to discrete states.
• Closed-Box Modeling (LLMs): To avoid the need for manual parameterization (e.g., defining cycle lengths), they explored "closed-box" approaches using Large Language Models (LLMs). The hypothesis was that LLMs could reason about ICS behavior without explicit parameter tuning.
• Dynamic Environment Simulation: To address the challenge of training data in modern, dynamic ICS (specifically those using wireless communication), the authors built a simulation environment. This allowed them to alter communication media (Wired vs. Wireless) without changing the underlying physical process, testing how timing-based detectors react to channel disturbances.

3. Key Challenges and Findings

The paper details three specific challenges encountered during the pursuit of a holistic IIDS:

Challenge 1: Discretization of Process States

• Issue: Converting continuous physical data into discrete labels for process mining is non-trivial. The quality of the resulting labels depends heavily on the underlying data distribution.
• Findings: The authors compared multiple discretization algorithms (Jenkspy, kMeans, Quantiles) against various performance metrics (Accuracy, Precision, Recall, F-scores, and eTa metrics).
  • Result: No single "best" discretization method exists. The optimal method varies depending on the specific performance metric prioritized (e.g., a method with high accuracy might have poor recall).
  • Implication: Selecting a discretization strategy requires careful trade-off analysis based on the specific security goals of the deployment.

Challenge 2: Parameterization and Interpretability

• Issue: Holistic models often require manual parameters (e.g., packet cycle lengths). The authors tested LLMs as a "parameterless" alternative.
• Findings:
  • Resource Constraints: Monitoring a complete ICS process with an LLM exceeded the capabilities of high-performance hardware (96GB VRAM), forcing a reduction to individual subprocesses.
  • Performance: When scoped to subprocesses, the LLM-based detector generated nearly constant alerts, failing to distinguish attacks from normal variance (except in low-variance scenarios).
  • Interpretability: As "closed-box" models, LLMs lack explainability. It is impossible to understand why an alert was triggered, making it difficult to distinguish true positives from false alarms.
  • Conclusion: Monolithic closed-box detectors are currently impractical for holistic IIDS due to high resource demands and lack of interpretability.

Challenge 3: Training Data in Dynamic Environments

• Issue: Modern ICS increasingly use wireless communication, which introduces dynamic behavior (noise, interference) that contradicts the assumption of deterministic ICS behavior.
• Findings: Using a simulated ICS, the authors compared Wired, Perfect Wireless, and Disturbed Wireless channels.
  • Variance vs. Mean: While the mean inter-arrival times in disturbed wireless channels were similar to wired channels, the standard deviation (variance) was substantially higher.
  • False Positives: Timing-based IIDS, which rely on deterministic intervals, cannot distinguish between a "distressed" wireless channel and a cyberattack. This leads to high false alarm rates or missed attacks if the model is not updated to reflect the new dynamic environment.
  • Implication: Training data collection is complicated by the fact that communication channels change over time, rendering static models obsolete quickly.

4. Key Contributions

1. Identification of Holistic Barriers: The paper moves beyond theoretical proposals to empirically demonstrate the practical difficulties of unifying network and physical process detection.
2. Discretization Analysis: Provides a comparative analysis showing that the choice of discretization method is metric-dependent, challenging the notion of a universal preprocessing step.
3. Evaluation of LLMs in ICS: Offers a critical assessment of LLMs for ICS security, highlighting their prohibitive resource costs and lack of interpretability in this specific domain.
4. Dynamic Channel Impact: Demonstrates experimentally that wireless channel disturbances mimic attack signatures in timing-based detectors, necessitating new detection mechanisms that account for environmental variance.

5. Significance and Future Directions

This paper serves as a reality check for the field of ICS security. It argues that while the concept of Holistic Intrusion Detection is theoretically sound and necessary for comprehensive security, current methodologies face significant practical hurdles.

• Shift in Research Focus: The authors conclude that future research must move away from static, deterministic models toward dynamic detection mechanisms. These new systems must be capable of adapting to changing environments (e.g., fluctuating wireless quality) and handling the inherent variability of modern industrial communication without generating excessive false positives.
• Practical Viability: The findings suggest that a "one-size-fits-all" holistic model is not yet feasible with current off-the-shelf techniques (like standard process mining or LLMs), urging the community to develop specialized, adaptive algorithms that can handle the interplay between network dynamics and physical process states.