🤖 machine learning

Behavioral Residualization for Unsupervised Intrusion Detection in Automotive CAN Networks

This paper introduces per-ID behavioral residualization, a CAN-specific representation that extracts and normalizes temporal, protocol, and payload features against individual arbitration ID baselines to significantly enhance unsupervised intrusion detection performance, particularly against attacks that reuse legitimate IDs, while explicitly quantifying its limitations in detecting novel-ID flooding and cross-ID fuzzing.

Original authors: Chandan Hegde, Mukundh R Reddy

Published 2026-08-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chandan Hegde, Mukundh R Reddy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your car is a bustling city where dozens of tiny computers, called Electronic Control Units (ECUs), are constantly chatting with each other to keep the engine running, the brakes working, and the radio playing. They talk over a single, shared highway called the CAN bus. This system was designed decades ago to be cheap and incredibly fast, so it skipped the security guards, the secret codes, and the locked doors that modern computers usually have. Because of this, if a hacker manages to plug into the car's diagnostic port or hack the infotainment system, they can shout fake messages onto that highway. They can tell the speedometer to say you're going 100 mph when you're stopped, or tell the brakes to lock up.

To stop this, security experts build "intrusion detection systems"—basically, digital bouncers that watch the traffic and yell "Stop!" when something looks weird. For a long time, these bouncers were very simple: they just looked for messages that had never been seen before, like a car with a license plate that doesn't exist. But smart hackers realized they could just steal a real license plate and use it to drive a stolen car. If the bouncer only checks for new plates, they miss the thief entirely. This paper asks a big question: Can we teach the bouncer to spot a thief even when they are wearing a perfectly legitimate disguise?

The authors of this paper, Chandan Hegde and Mukundh R Reddy, propose a new way to watch the car's traffic called "Behavioral Residualization." Instead of just checking if a message is new, they teach the system to know the "personality" of every single message ID. Think of it like a parent who knows their child's daily routine so well that they can tell immediately if the child is acting strangely, even if the child is wearing the same clothes and using the same name.

The researchers built a system that watches each specific message ID (like the one for the speedometer) and learns its normal habits: how fast it usually talks, how long its messages are, and what the data inside looks like. They then create a "baseline" for what is normal. When a new message arrives, the system doesn't ask, "Have I seen this before?" Instead, it asks, "Is this message behaving differently than this specific ID usually does?" If the speedometer suddenly starts sending messages twice as fast as it ever has, the system flags it as a "behavioral residual"—a weird deviation from the norm.

The paper tested this idea on two different sets of car data. One was a synthetic dataset (HCRL) where hackers used obvious tricks, and the other was a much harder, real-world dataset (ROAD) where hackers reused legitimate IDs to hide their tracks. The results were impressive. When the hackers tried to reuse a real ID to mess with the speedometer or the engine temperature, this new "behavioral" system caught them almost every time, with a success rate (recall) of 99% or higher. It worked so well that it didn't matter which specific "detector" algorithm they used; the magic was in the way they represented the data.

However, the authors are very honest about where their system doesn't work. They found that if a hacker floods the network with thousands of brand-new, fake IDs all at once (a "DoS" attack), or if they mess with a huge group of different IDs simultaneously (a "fuzzing" attack), the system gets confused. Because the system is so focused on the individual habits of each ID, it misses the big picture when the whole crowd goes crazy. It's like a bouncer who is so good at spotting one specific person acting weird that they miss a riot happening in the lobby.

In the end, the paper shows that by focusing on how a message behaves rather than just what it is, we can catch sophisticated hackers who try to blend in. While the system isn't perfect against every type of attack, it provides a much stronger shield against the realistic threat of hackers using stolen identities, which is exactly what modern regulations are worried about. The authors suggest this approach is a solid starting point for building smarter, more adaptive security for our future cars.

Technical Summary: Behavioral Residualization for Unsupervised Intrusion Detection in Automotive CAN Networks

1. Problem Statement

Modern vehicles rely on the Controller Area Network (CAN) bus, a protocol designed for low cost and real-time performance that lacks native message authentication, encryption, or integrity protection. Consequently, an attacker with physical or remote access (e.g., via a compromised diagnostic port or infotainment unit) can inject arbitrary frames.

While regulatory frameworks like UNECE R155 and ISO/SAE 21434 mandate intrusion detection systems (IDS), most existing solutions rely on presence-based features. These methods detect anomalies by identifying novel arbitration IDs, frozen payload bytes, or unusual Data Length Code (DLC) values. While these approaches achieve near-perfect scores on public datasets like HCRL (which contain trivially separable presence attacks), they fail against realistic adversaries who reuse legitimate arbitration IDs. In such scenarios, surface-level anomalies are absent, and detection must rely on subtle behavioral deviations within the traffic of known IDs.

2. Methodology: Per-ID Behavioral Residualization (PIRD)

The paper proposes Per-ID Behavioral Residualization (PIRD), a representation learning technique designed specifically for the challenge of detecting attacks that reuse legitimate IDs. Rather than proposing a new detection algorithm, PIRD transforms the input data into a representation that highlights deviations from an ID's specific normal behavior.

Core Mechanism

Sliding Window Feature Extraction: For each arbitration ID $k$ , the system maintains a sliding window of the most recent $w$ frames (default $w=30$ ). From these windows, 14 behavioral features are extracted across three families:
- Inter-arrival Time (IAT): Mean, standard deviation, median, min, and max.
- DLC: Mode and standard deviation of the data-length code.
- Payload: Mean, variance, max absolute consecutive change, number of distinct byte values, and Shannon entropy.
- Context: Window fill ratio and time since the ID was last observed.
Per-ID Residual Transform: The core contribution is the residualization step. During a calibration phase using only benign traffic, the system computes the mean ( $\mu^k_j$ ) and standard deviation ( $\sigma^k_j$ ) for each feature $j$ specific to arbitration ID $k$ . For any test window, the raw feature value $x^k_{t,j}$ is transformed into a standardized residual:
$r^k_{t,j} = \frac{x^k_{t,j} - \mu^k_j}{\sigma^k_j + \epsilon}$
If an ID is absent from calibration, global statistics are used as a fallback.
Detector-Agnostic Evaluation: The resulting residual vectors are fed into standard unsupervised anomaly detectors (Isolation Forest, One-Class SVM, LOF, HBOS, Elliptic Envelope, and Autoencoders). The framework is designed so that the representation, not the specific detector, drives performance gains.
Per-Capture Evaluation Protocol: For the ROAD dataset, which consists of independent driving sessions, the authors introduce a "per-capture" protocol. Calibration and thresholding are performed on the pre-injection normal segment of each individual capture to ensure temporal alignment, rather than treating the dataset as a single continuous stream.

3. Key Contributions

A Per-ID Behavioral Residual Representation: A method that conditions every feature on an arbitration ID's own normal baseline. This ensures that an injected frame on a legitimate ID produces a large standardized deviation, even if the ID itself is not new.
Multi-Detector Validation: Evidence demonstrating that the performance gains are inherent to the residual representation rather than a specific algorithm. The method was evaluated across six unsupervised detectors.
Rigorous Evaluation on Realistic Data: Application of the method to the ROAD dataset, where attacks reuse legitimate IDs and are physically verified, contrasting with the "presence-biased" HCRL dataset.
Explicit Robustness Boundaries: Honest quantification of failure modes (novel-ID flooding and cross-ID fuzzing) rather than treating them as hidden defects.

4. Experimental Results

Performance on HCRL (Synthetic/Presence-Based)

Residual vs. Raw: Raw behavioral features performed poorly (F1 $\approx$ 0.00 for RPM and Gear spoofing). Residualization dramatically improved performance, achieving F1 scores of 0.992 for RPM spoofing and 0.946 for gear spoofing.
Detector Independence: Across five random seeds and six detectors, residualization improved the mean F1 in 21 of 24 cells.
Failure Modes: The method failed on DoS attacks involving novel IDs (F1 = 0.02) because the per-ID baseline could not be established for unknown IDs.

Performance on ROAD (Real-World/Legitimate-ID Reuse)

Targeted Attacks: For targeted single-AID attacks (correlated-signal, speedometer, reverse-light), PIRD achieved recall $\ge$ 0.99 with ROC-AUC $\approx$ 0.99, even though the attacks reused legitimate IDs.
Fuzzing: Performance dropped significantly on cross-ID fuzzing attacks (F1 $\approx$ 0.27), as these represent a population-level shift that per-ID conditioning cannot detect.
False Positive Rate (FPR): The residual FPR was observed between 2–5% on ROAD, attributed to benign driving-state variations (e.g., throttle changes) shifting normal windows above the threshold.

Efficiency

Latency: Detection delay is concentrated at zero windows; an attack is flagged within a single sliding-window step (approx. 1.1 ms worst-case).
Throughput: The system processes roughly 0.2 million windows per second on commodity hardware, with negligible overhead added by the residual transform.

5. Significance and Claims

The paper claims that per-ID behavioral residualization is a generally useful representation for CAN intrusion detection, distinct from any specific detection algorithm.

Shift from Presence to Behavior: The work demonstrates that effective IDS for modern threats must move beyond checking for "new" IDs or "frozen" bytes. Instead, it must detect deviations in the behavior of known IDs.
Generalization: The method successfully generalizes from synthetic data (HCRL) to physically verified, realistic data (ROAD), proving that the gains are not artifacts of dataset-specific shortcuts.
Defined Coverage: The authors explicitly define the method's "coverage boundary." PIRD is highly effective for targeted legitimate-AID attacks but is blind to novel-ID flooding and cross-ID population shifts. The paper argues that this limitation is a necessary trade-off for robustness against legitimate-ID reuse, rather than a failure of the system.
Reproducibility: The study establishes a per-capture evaluation protocol necessary for session-structured real-world data and provides open-source code and configurations to support reproducibility.

In conclusion, the paper posits that the residual operator is the decisive component that enables unsupervised detection of sophisticated CAN intrusions, provided the threat model involves the reuse of legitimate arbitration IDs.