Drift-Aware RL-based Wavelet Denoising for Network-Traffic Anomaly Detection

Technical Summary: Drift-Aware RL-based Wavelet Denoising for Network-Traffic Anomaly Detection

Problem Statement
Network traffic monitoring relies on fine-grained utilization measurements to drive operational decisions, specifically anomaly detection (identifying short-lived load excursions) and capacity estimation (quantifying the 95th percentile of utilization, $C_{95}$). However, telemetry data is corrupted by two distinct factors: stationary background noise and statistical drift. Drift refers to time-dependent changes in the signal's mean, variance, distributional shape, or tail behavior.

Traditional wavelet denoising methods (e.g., VisuShrink, SureShrink) rely on static thresholds calibrated under stationary, independent and identically distributed (i.i.d.) Gaussian assumptions. Under drift, these static rules become mismatched. At moderate-to-high signal-to-noise ratios (SNR), aggressive fixed thresholds can over-suppress useful signal structure, erasing the very transient bursts required for anomaly detection and biasing the upper-tail statistics needed for capacity estimation. Consequently, optimizing for reconstruction fidelity (minimizing Mean Squared Error) often degrades downstream task performance.

Methodology
The authors propose a drift-aware framework that treats adaptive wavelet denoising as a preprocessing layer optimized for downstream task utility rather than signal reconstruction fidelity. The system operates in two stages:

1. Drift Detection Gate:
   Before denoising, a windowed signal is analyzed by a four-detector ensemble to determine if the statistical regime has shifted. The detectors are:

   • Page-Hinkley (PH): Detects mean shifts.
   • Variance-Ratio: Detects variance changes (heteroscedasticity).
   • Jensen-Shannon Divergence (JSD): Detects structural divergence (distributional shape changes).
   • Anderson-Darling (AD): Detects tail amplification (impulsive noise).

   These detectors produce binary flags and continuous statistics, which are fused via a logical OR to create a gate signal ($G_m$). If no drift is detected, a fixed baseline denoiser (VisuShrink) is applied. If drift is detected, a Reinforcement Learning (RL) agent is invoked.

2. RL-Based Adaptive Denoising:
   The adaptive component is formulated as a Markov Decision Process (MDP) solved using Proximal Policy Optimization (PPO).

   • State Space: Includes signal descriptors (mean, variance, MAD), change-sensitive descriptors (relative changes from history), and the full detector evidence vector (flags and statistics).
   • Action Space: A mixed discrete-continuous space selecting the wavelet configuration: mother wavelet, decomposition depth, thresholding function (soft/hard), threshold selection strategy, a continuous threshold gain, and a per-scale shaping parameter.
   • Reward Function: Unlike prior work, the reward is task-utility driven. It is a convex combination of:
     • The gain in anomaly detection F1 score (comparing denoised vs. raw signal against ground truth bursts).
     • The reduction in capacity estimation error ($| \hat{C}_{95} - C_{95} |$).

   The anomaly target is defined on the clean signal, and the drift gate operates on the corruption, ensuring the stages are non-circular.

Key Contributions

• Task-Aware Formulation: The paper formulates adaptive wavelet denoising as a sequential decision problem optimized for specific downstream monitoring tasks (detection and capacity estimation) rather than generic reconstruction fidelity.
• Complementary Drift Gate: A novel four-detector gate (PH, VAR, JSD, AD) allows the RL policy to condition on the presence, type, and intensity of drift, enabling targeted adaptation.
• Discriminating Task Design: By defining anomalies as multi-scale transient bursts (which low-pass filters remove but wavelets preserve), the framework explicitly distinguishes wavelet denoising from naive smoothing.
• Mixed Action Space RL: The integration of PPO over a mixed discrete-continuous action space allows for fine-grained control over wavelet parameters (mother wavelet choice, depth, thresholding strategy, and continuous gain).
• Comprehensive Benchmarking: The method is benchmarked against a low-pass moving-average filter, classical shrinkage rules (VisuShrink, SureShrink, BayesShrink), and a Wiener filter across four drift types and a wide range of input SNRs.

Experimental Results
The framework was evaluated on synthetic traffic-utilization data augmented with four drift types (mean shift, variance change, structural divergence, tail amplification) across SNRs from -10 dB to 30 dB.

• Drift Detection: The fused gate achieved a 91.54% true-positive rate for drift detection with a low false-alarm rate (1.23% on no-drift controls), effectively routing signals to the appropriate denoising branch.
• Denoising Performance (SNR Improvement): The RL policy consistently outperformed the fixed VisuShrink baseline in terms of SNR improvement ($\Delta$SNR). Notably, at high input SNRs (20–30 dB), the fixed baseline became counter-productive (negative $\Delta$SNR), destroying signal energy, whereas the RL policy remained beneficial or caused only mild degradation.
• Anomaly Detection: The proposed method achieved the highest Area Under the Curve (AUC) for transient anomaly detection on three of the four drift types among all denoising methods, approaching the performance of the oracle (clean signal). It significantly outperformed low-pass filters, which smeared the transient bursts.
• Capacity Estimation: The RL policy yielded the lowest error in estimating $C_{95}$ across all drift types and SNRs. Fixed shrinkage rules tended to either over-provision (due to noise inflation) or under-provision (due to over-smoothing), while the adaptive policy tracked the upper tail more accurately.
• Statistical Significance: Paired Wilcoxon signed-rank tests confirmed that the proposed method's improvements in both AUC and capacity error were statistically significant compared to all baselines.

Significance and Claims
The paper claims that optimizing denoising for downstream task utility is superior to optimizing for reconstruction fidelity in non-stationary network environments. The primary significance lies in the demonstration that a learned policy can dynamically adapt to statistical drift, avoiding the "one-size-fits-all" failure of static wavelet rules. Specifically, the framework prevents the over-smoothing of genuine load dynamics at high SNR, a critical failure mode for fixed thresholding methods. The authors note that while the current results use a synthetic stand-in for the final policy evaluation (due to data availability constraints at the time of writing), the methodology and drift-detection results are derived from the original experimental pipeline, and the framework is designed to be validated on public real-world traffic telemetry in future work.