💻 computer science

Between Zeros and Ones: Behavioral Characterization Beyond Binary Labeling Across Public ICS Datasets

This paper proposes a behavioral characterization framework that maps ICS process traces into five physical primitives to reveal significant behavioral diversity and cross-dataset biases masked by traditional binary labeling, arguing for behavior-stratified evaluation to expose performance blind spots and better support incident response.

Original authors: Konstantinos E. Kampourakis, Vyron Kampourakis, Georgios Spathoulas, Constantinos Kolias

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Konstantinos E. Kampourakis, Vyron Kampourakis, Georgios Spathoulas, Constantinos Kolias

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a security guard for a massive, complex factory. Your job is to watch the control room screens and shout "Emergency!" if something goes wrong.

For a long time, security researchers have given you a very simple rulebook: "If the screen looks weird, it's an Attack. If it looks normal, it's Normal." They call this a "binary" (two-choice) label.

The authors of this paper argue that this rulebook is too simple. It's like a doctor who only asks, "Do you have a fever?" without ever asking what kind of sickness you have. A fever could be from the flu, a broken thermometer, or a hot day. In a factory, a "weird screen" could be a valve slamming shut, a sensor slowly drifting off, or a pump getting stuck in a loop.

Here is the paper's story, broken down into simple concepts:

1. The Problem: The "One-Size-Fits-All" Label

The researchers looked at three famous "training factories" (datasets called SWaT, WADI, and HAI) used to teach computers how to spot attacks. They noticed that everyone treats every attack as the exact same thing: BAD.

But in reality, industrial attacks are like different types of weather:

Drift: A slow, gradual change (like a slow leak).
Spike: A sudden, violent jolt (like a lightning strike).
Oscillation: A shaking or wobbling motion (like a car engine misfiring).
Repetition: A stuck record playing the same note over and over.
Switching: A light switch being flipped on and off too fast.

If you just tell the computer "It's an attack," the computer doesn't know if it needs to shut down the whole factory immediately (for a spike) or just schedule a repair for later (for a drift).

2. The Solution: A New "Behavioral Lens"

The authors built a new tool—a Behavioral Characterization Framework. Think of this tool as a pair of special glasses that doesn't just see "Attack vs. Normal," but instead sees how the attack is behaving.

They broke down every moment of data into five simple "primitives" (basic building blocks):

Drift: Is it slowly sliding away?
Spike: Did it jump suddenly?
Oscillation: Is it wobbling back and forth?
Repetition: Is it repeating a pattern?
Switching: Is it flipping states rapidly?

They applied these glasses to the three famous datasets to see what they actually looked like.

3. The Discovery: The Factories Are Different

When they looked through their new glasses, they found that the three datasets were actually very different neighborhoods:

The HAI Dataset: This one was full of Spike and Drift attacks. It was like a neighborhood where people were constantly slamming doors and slowly moving furniture.
The SWaT Dataset: This one was dominated by Oscillation (wobbling) and a lack of repetition. It was like a neighborhood where things were shaking and freezing, but not repeating patterns.
The WADI Dataset: This one was mostly Repetition. It was like a neighborhood where everything was stuck on a loop.

The Big Reveal: If you train your security guard (AI model) only on the HAI dataset, they will get really good at spotting "Spike" attacks but might miss the "Oscillation" attacks that happen in the SWaT dataset. The current way of testing (just saying "Attack" or "Not Attack") hides these differences.

4. The Test: Does the "Simple" Label Lie?

To prove their point, the researchers ran a simple test. They took a standard security AI and asked it two questions:

Question A (The Old Way): "Is this an attack?" (Yes/No)
Question B (The New Way): "Is this a Drift, a Spike, an Oscillation, a Repetition, or a Switch?"

The Results:

Question A: The AI scored very high (around 85% accuracy). It looked like a genius.
Question B: The AI's score crashed (dropping to around 37-42%).

What this means: The AI was good at spotting that something was wrong, but it was terrible at figuring out what kind of wrong it was. The "simple" score was hiding the fact that the AI couldn't tell the difference between a slow leak and a sudden explosion.

5. The Conclusion: We Need More Than Just "Yes/No"

The paper concludes that we shouldn't stop using the simple "Attack/Normal" labels entirely. They are still useful as a first alarm.

However, relying only on that simple score is dangerous. It's like a weather report that only says "Storm" without telling you if it's a hurricane, a tornado, or a hailstorm.

The authors suggest we need to add a second layer of evaluation: Behavior-Stratified Evaluation. This means we should check if our security systems can actually distinguish between the different types of bad behavior, not just detect that bad behavior exists. This helps operators know exactly how to react when an alarm goes off.

In short: The paper argues that we are currently judging industrial security systems by how well they shout "Fire!" but we aren't checking if they know the difference between a grease fire and a gas explosion. We need to teach them to tell the difference.

Technical Summary: Between Zeros and Ones: Behavioral Characterization Beyond Binary Labeling Across Public ICS Datasets

Problem Statement
Current Intrusion Detection Systems (IDS) for Industrial Control Systems (ICS) rely heavily on public benchmarks such as SWaT, WADI, and HAI. These datasets are typically evaluated using binary labels ("normal" vs. "attack") and aggregate metrics like precision, recall, and F1-score. This practice implicitly assumes the "attack" class is homogeneous. However, cyber-physical attacks manifest through diverse temporal behaviors—such as gradual sensor drifts, abrupt spikes, frozen telemetry, oscillatory control loops, and forced actuator switching. Binary labeling collapses these qualitatively different urgencies into a single signal, potentially masking behavioral diversity and failing to provide the specificity required for differentiated operator responses (e.g., immediate safety trips vs. long-term recalibration). Furthermore, existing evaluations often overlook dataset bias, where specific benchmarks may overrepresent certain attack morphologies while underrepresenting others.

Methodology
The authors propose a purely observational and exploratory framework to characterize ICS attack traces using five interpretable physical primitives, without inferring attacker intent or semantic attack types. The methodology proceeds through four stages:

Preprocessing and Windowing: Multivariate ICS traces (continuous sensors and discrete actuators) are normalized using z-score normalization based on normal operating data to ensure equitable contribution across variables. The data is segmented into overlapping temporal windows (60 samples, 50% overlap). Windows are labeled as "attack" if at least 50% of their constituent samples are attack-labeled in the ground truth.
Behavioral Primitive Extraction: For each window, five lightweight, computationally inexpensive primitives are calculated based on elementary time-series statistics:
- Drift: Sustained monotonic deviation (linear slope magnitude $\times$ regression fit quality).
- Spike: Abrupt transient change (maximum absolute first-order temporal difference).
- Oscillation: Repeated directional reversals (normalized frequency of sign changes in the first derivative).
- Repetition: Recurring temporal patterns (maximum autocorrelation over a bounded lag interval).
- Switching: Excessive discrete state transitions (count of state changes for actuator signals).
Feature Vector Construction: To capture localized attack effects without being skewed by isolated noise, the primitive scores across all process variables are aggregated using the 95th percentile operator, resulting in a five-dimensional feature vector per window.
Dominant Property Assignment: For visualization and coarse analysis, each window is assigned a single "dominant" behavioral property based on the highest normalized primitive activation, provided it exceeds a threshold ( $\tau = 0.30$ ).

Key Contributions

Behavioral Characterization Framework: A portable, domain-agnostic method to map raw multivariate process traces into five interpretable temporal primitives, exposing the behavioral diversity often hidden by binary labels.
Dataset Bias Analysis: An empirical demonstration that SWaT, WADI, and HAI occupy distinct regions of the behavioral space, indicating that benchmarking on a single dataset emphasizes specific behavioral profiles while potentially underrepresenting others.
Evaluation Implications: A comparative analysis showing that aggregate binary metrics can obscure performance variations across different behavioral proxies. The authors introduce a behavior-stratified evaluation approach to complement standard binary benchmarking.

Results
The framework was applied to SWaT, WADI, and HAI, addressing four research questions:

RQ1 (Behavioral Shifts): Attack windows exhibit measurable behavioral shifts relative to normal operation, but the nature of the shift is dataset-dependent. For instance, HAI attacks show significant increases in drift and spike, while SWaT attacks are characterized by increased oscillation and decreased repetition.
RQ2 (Dataset Differences): The datasets populate the behavioral space differently. HAI is dominated by spike and drift; WADI is characterized by repetition; and SWaT is defined by oscillation and "frozen-telemetry" behavior (reduced repetition).
RQ3 (Intra-Dataset Diversity): Even within a single dataset, attack events are not homogeneous. Attack windows often exhibit co-occurring behaviors (e.g., simultaneous spike and drift in HAI), which binary labels collapse into a single class.
RQ4 (Evaluation Impact): Using a Random Forest baseline, the authors found that while binary classification yields high macro F1 scores (e.g., 85.44% on SWaT), performance drops significantly when the task is reformulated as multiclass classification based on behavioral proxies (dropping to 37.84% on SWaT). This suggests that while models may detect "anomalies," they struggle to uniformly distinguish between different types of behavioral manifestations.

Significance and Claims
The paper argues that binary detection is a necessary first stage but insufficient for robust ICS security evaluation. The authors do not claim that binary evaluation hides detection failures per se, but rather that it obscures how performance varies across different behavioral manifestations. By complementing conventional binary benchmarking with behavior-stratified evaluation, the framework aims to:

Expose "blind spots" that aggregate scores leave hidden.
Reveal cross-dataset bias and intra-dataset diversity.
Better support targeted incident response by providing behavioral specificity.

The authors position this work as a "portable first step" toward understanding behavioral diversity, emphasizing that the framework describes observable temporal morphology rather than inferring semantic attack taxonomies or attacker intent. They acknowledge limitations, including the inability to capture purely cyber attacks that do not affect physical variables and the potential for the single-label assignment to obscure mixed behaviors. Future work is suggested to expand the behavioral vocabulary, move toward multi-label representations, and integrate these characterizations into operational triage and incident response workflows.