A Double Proportionate Sparse Adaptive Filter for Impulsive Noise Environments

Technical Summary: Double Proportionate Sparse Adaptive Filter (DP-SAF)

Problem Statement
Adaptive filters are essential for system identification, channel estimation, and acoustic echo cancellation. However, existing algorithms face a trade-off gap when dealing with systems that are both sparse (having few active coefficients) and subject to impulsive noise.

1. Sparsity: Standard Least Mean Square (LMS) algorithms assign equal gain to all coefficients, which is inefficient for sparse systems where adaptation energy should focus on active taps. Proportionate Normalized LMS (PNLMS) addresses this but lacks robustness against impulsive noise.
2. Impulsive Noise: Algorithms robust to impulsive noise (e.g., those using sign-error updates or M-estimators) often fail to exploit sparsity or lack mechanisms to actively suppress inactive coefficients.
3. The Gap: No existing algorithm simultaneously integrates independent proportionate adaptation for active taps, zero-attraction penalties for inactive taps, and impulsive noise robustness within a single update step of linear complexity $O(M)$.

Methodology
The paper proposes the Double Proportionate Sparse Adaptive Filter (DP-SAF), which updates the weight vector $w(n)$ using two independent diagonal gain matrices, $G_1$ and $G_2$, alongside a sign-error update. The update equation is:
$$w(n + 1) = w(n) + \mu G_1(n) x(n) \text{sgn}(e(n)) - \rho G_2(n) \text{sgn}(w(n))$$

• Sign-Error Update: Replacing the raw error $e(n)$ with $\text{sgn}(e(n))$ limits the influence of large outliers, providing robustness against impulsive corruptions.
• Gain Matrix $G_1$ (Proportionate Adaptation): This matrix scales the gradient step based on coefficient magnitudes. It is designed to accelerate convergence for active taps (large coefficients) while maintaining a non-zero baseline for inactive ones. It is derived as a normalized proportionate form:
  $$g_{1,k}(n) = \frac{\max(\delta, |w_k(n)|)}{\frac{1}{M}\sum_{j=1}^M \max(\delta, |w_j(n)|)}$$
• Gain Matrix $G_2$ (Zero-Attraction): This matrix applies a penalty to pull coefficients toward zero. Unlike $G_1$, it is designed to be strongest for small (likely inactive) coefficients and weakest for large (active) ones, preventing the suppression of active taps. It follows a magnitude-dependent reduction:
  $$g_{2,k}(n) = \frac{1}{1 + \varepsilon |w_k(n)|}$$
• Derivation: Both gain matrices are derived from a minimum-norm optimization framework that minimizes a cost function balancing steady-state estimation error and weight perturbation. The design ensures $G_1$ and $G_2$ work complementarily rather than antagonistically.
• Complexity: The algorithm maintains $O(M)$ complexity per iteration, identical to LMS, by avoiding matrix inversions through its diagonal structure.

Key Contributions

1. Dual Proportionate Mechanism: DP-SAF is the first algorithm to apply independent proportionate weighting to both the gradient step ($G_1$) and the zero-attraction term ($G_2$) in a single update.
2. Optimization-Based Design: The gain matrices are derived from a minimum-norm framework, ensuring they follow opposite proportionality rules by design (one amplifies large coefficients, the other suppresses small ones).
3. Robustness and Efficiency: The sign-error update bounds the impact of impulsive noise, while the diagonal structure ensures the algorithm is suitable for real-time and embedded implementations without sacrificing convergence speed for robustness.

Simulation Results
The authors evaluated DP-SAF against seven competing algorithms (LMS, PNLMS, RZA-LMS, VSS-RZA-LMM, GMN, GCMPN, LCMN) using a sparse FIR filter ($M=32$, 10% active taps) under a Bernoulli impulsive noise model ($p=0.2$, high amplitude).

• Steady-State Accuracy: In Experiment 1, DP-SAF achieved the lowest steady-state Mean Square Deviation (MSD) floor among all competitors, demonstrating that the dual-gain design improves accuracy without trading off convergence speed.
• Convergence Speed: In Experiment 2, when all algorithms were allowed to converge to the same steady-state MSD, DP-SAF reached that level significantly faster than the others.
• Conclusion from Data: The results confirm that the dual-gain structure improves both transient speed and steady-state accuracy simultaneously, avoiding the typical trade-offs seen in other methods.

Significance and Claims
The paper claims that DP-SAF fills a critical gap in adaptive filtering by unifying sparsity exploitation, zero-attraction regularization, and impulsive noise robustness. The authors emphasize that the algorithm achieves this without increasing computational complexity beyond the standard LMS level. The significance lies in the "complementary by construction" nature of the two gain matrices, which reinforce one another rather than competing, allowing the filter to handle sparse, time-varying systems in harsh noise environments effectively. The paper concludes by noting that extending DP-SAF to complex-valued systems and inputs with strong spectral coloring remains an open direction for future work.