🔭 astrophysics

Real-Time RFI Mitigation in SPOTLIGHT: A Two-Stage Approach for Transient Searches

This paper presents a real-time, two-stage RFI mitigation framework for the SPOTLIGHT system at the uGMRT, combining antenna-level voltage filtering (VOLT) and statistical time-domain processing (STRIPE) to significantly reduce false detections and improve signal-to-noise ratios for radio transient searches.

Original authors: Raghav Wani, Jayanta Roy, Harshavardhan Reddy, Sanjay Kudale, Ujjwal Panda, Karel Adamek, Wesley Armour, Kshitij Bane, Kaushal Buch, Jayaram Chengalur, Jyotirmoy Das, Sridhar Gajendran, SPOTLIGHT Coll

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Raghav Wani, Jayanta Roy, Harshavardhan Reddy, Sanjay Kudale, Ujjwal Panda, Karel Adamek, Wesley Armour, Kshitij Bane, Kaushal Buch, Jayaram Chengalur, Jyotirmoy Das, Sridhar Gajendran, SPOTLIGHT Collaboration

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The universe speaks in radio waves, a silent language carried by particles that have traveled across vast distances to reach our telescopes. To hear this language clearly, astronomers point massive dishes at the sky, listening for faint signals from pulsars, the spinning remnants of dead stars, or the mysterious, fleeting flashes known as fast radio bursts. However, the Earth is a noisy place. Our own technology—cell phones, power lines, satellites, and even wind turbines—creates a constant static that drowns out the cosmic whispers. This interference, known as radio frequency interference, is a major obstacle. It hides the very signals scientists are desperate to find. For a telescope to work effectively, it must be able to distinguish between the noise of human civilization and the genuine voice of the cosmos, all while processing data at the speed of light.

A team of researchers working with the upgraded Giant Metrewave Radio Telescope in India has developed a new way to solve this problem. They created a system called SPOTLIGHT, designed to hunt for radio transients in real time. The challenge was that the telescope generates so much data that it cannot wait to analyze it later; it must filter out the noise instantly as the data arrives. If the system is overwhelmed by false alarms caused by human-made interference, it would run out of storage space in a matter of days, forcing the telescope to stop listening. To prevent this, the team built a two-stage cleaning process that acts like a highly efficient sieve, removing different types of noise at different points in the data stream.

The first stage of this cleaning happens before the telescope even combines the signals from its individual antennas. Here, a module called VOLT scans the raw electrical voltages coming directly from the dishes. It looks for sudden, sharp bursts of energy that look like static from a lightning strike or a switching power supply. When it finds these impulsive bursts, it removes them immediately. This is crucial because if these bursts are allowed to pass through, they would get amplified and mixed into the final image, making them much harder to get rid of later. By catching this noise early, the system prevents it from spreading through the entire dataset.

The second stage, called STRIPE, works on the data after the telescope has combined the signals from all its antennas into a focused beam. This stage looks for a different kind of trouble: steady, narrow hums from radio stations or slow, drifting changes in the background signal that can mask faint cosmic pulses. STRIPE uses statistical methods to identify these patterns. It knows what a clean signal should look like and flags anything that deviates from that pattern. If it finds a frequency channel that is constantly buzzing or a time slice that is unnaturally flat, it replaces that data with a neutral, random noise that preserves the overall shape of the signal without the interference. This ensures that the underlying astrophysical signal remains intact while the human-made noise is stripped away.

The results of this new approach are dramatic. Before the system was fully deployed, the telescope was generating so many false alarms that it would have filled its entire storage capacity in just over two days. After the two-stage filter was turned on, the number of false alarms dropped by 98 percent. This reduction allowed the system to run continuously for months without running out of space, transforming it from a storage-limited experiment into a reliable, long-term observatory. The researchers tested the system by observing known pulsars, which serve as a reliable benchmark. They found that the cleaning process did not just remove noise; it actually made the real signals clearer. The strength of the recovered pulsar signals improved by a factor of 2.7 compared to the unfiltered data.

Furthermore, the team compared their new real-time filter against a standard, offline tool used by astronomers for years. They found that their new system performed just as well, and in most cases, better, at improving the clarity of the signals. Most importantly, the system proved capable of recovering genuine single pulses from pulsars that were completely hidden in the noisy, unfiltered data. In one specific test, the number of detected pulses increased by more than 400 percent after the full cleaning process was applied. This means the telescope is now seeing things it was previously blind to.

The success of this project demonstrates that it is possible to build a real-time system that can handle the immense volume of data from a modern radio telescope without sacrificing sensitivity. By catching noise at two different stages of the process, the SPOTLIGHT team has ensured that the telescope can listen to the universe with unprecedented clarity. This capability is essential for the future of radio astronomy, where the goal is to catch the rarest and faintest signals from the cosmos before they fade away. The system is now in routine operation, ready to discover new radio transients that were previously lost in the static of our own world.

Technical Summary: Real-Time RFI Mitigation in SPOTLIGHT

Problem Statement
Radio Frequency Interference (RFI) poses a critical threat to the sensitivity and reliability of modern radio transient surveys, particularly for real-time searches of fast radio transients. As human-made interference sources (e.g., telecommunications, wind turbines, satellite constellations) proliferate, they degrade data quality and generate false triggers. While sophisticated offline RFI mitigation algorithms exist, they are often computationally prohibitive for real-time applications. Real-time systems face strict constraints: they must process data streams in a single pass with limited data access, requiring algorithms that are both statistically robust and computationally efficient to maintain low latency. The SPOTLIGHT system, a commensal real-time transient search backend at the upgraded Giant Metrewave Radio Telescope (uGMRT), requires a solution that mitigates RFI without relying on the system's limited GPU resources, which are dedicated to the transient search pipeline.

Methodology
The authors present a two-stage, real-time RFI mitigation framework designed specifically for the SPOTLIGHT backend. This framework operates at complementary stages of the signal processing chain to address different classes of interference while adhering to stringent latency requirements.

Stage 1: VOLT (Voltage-Level Filtering)
- Operation: VOLT operates on Nyquist-sampled voltage time series from individual antennas prior to correlation and beamforming.
- Algorithm: It utilizes a Median-of-MAD (Median Absolute Deviation) estimator. Instead of calculating thresholds from a single data window, it computes the median of MAD estimates across successive windows ( $k=20,480$ ). This approach provides robustness even when up to 50% of the data is contaminated.
- Implementation: Adapted from the GMRT Wideband Backend (GWB), VOLT is implemented as a multi-CPU application. It processes dual-polarization complex voltage streams from 32 antennas (64 inputs) distributed across 16 nodes.
- Constraints: Currently limited to 100 MHz and 200 MHz bandwidth modes due to computational and memory constraints; it is bypassed for 400 MHz operations.
Stage 2: STRIPE (Beamformed Data Filtering)
- Operation: STRIPE (SPOTLIGHT Time-domain RFI Processing Engine) operates on beamformed dynamic spectra after correlation, specifically during the transfer of data from the smaller TELRing buffer to the larger FRBRing buffer.
- Algorithm: STRIPE employs a three-step statistical filtering process adapted from PulsarX's filtool:
  - Patch Filter: Identifies time samples with zero variance across the entire band (indicative of impulsive broadband RFI) and replaces them with Gaussian noise.
  - Skewness-Kurtosis (SK) Filter: Detects narrowband RFI by calculating spectral skewness, excess kurtosis, and autocorrelation (ACF1). Outliers are identified using Tukey's fences based on the Interquartile Range (IQR).
  - Baseline Filter: Removes slow-varying baseline fluctuations using a running median filter and linear least-squares fitting.
- Implementation: STRIPE is a lightweight, CPU-based sub-module of the Ares utility. It processes 640 post-correlation (PC) beams across 256 CPU cores, achieving a processing speed approximately 3.4 times faster than real-time.

Key Contributions

Framework Development: The creation of a complete, two-stage real-time RFI mitigation system (VOLT + STRIPE) that operates entirely on CPU resources, preserving GPU capacity for the transient search pipeline.
Algorithmic Adaptation: The successful adaptation of offline statistical methods (MAD estimators, Tukey's fences) into a real-time, single-pass architecture with specific optimizations for the SPOTLIGHT shared-memory ring buffer system.
Parameter Optimization: Systematic optimization of STRIPE's tunable parameters (IQR threshold $\kappa$ and baseline window size $\omega$ ) using a dataset of 70 pulsar observations, establishing default values ( $\kappa=4.0, \omega=0.5$ ) that yield near-optimal performance across diverse observing conditions.

Results
The framework was evaluated using routine commensal observations, controlled pulsar observations, and benchmarking against PRESTO's rfifind.

False Trigger Reduction: The deployment of the two-stage framework reduced the median real-time trigger rate from ~5 triggers per 10 minutes (without mitigation) to ~0.09 triggers per 10 minutes. This represents a 98% reduction in the false detection rate, extending the projected storage exhaustion time from 2.3 days to 125.5 days, thereby enabling sustained operations within the system's design limits.
Signal-to-Noise (S/N) Improvement:
- Folded S/N: In a comparison with rfifind, STRIPE demonstrated superior or comparable performance, achieving a median relative S/N improvement 17.6% larger than rfifind across 55 pulsar observations. STRIPE outperformed rfifind in 76.4% of cases.
- Single Pulse Recovery: For single-pulse searches, the two-stage filtering resulted in a 2.7 $\times$ improvement in the median S/N of recovered astrophysical pulses compared to unfiltered data.
- Pulse Recovery Rates: The number of genuine single pulses recovered increased significantly. For Incoherent Array (IA) beams, the VOLT+STRIPE configuration increased the number of detected genuine pulses by 456% (Band 3) and 122% (Band 5) relative to unfiltered data.
Computational Efficiency: STRIPE processes data at a mean real-time factor of 3.38, ensuring that the buffer (TELRing) is never exhausted and no data is lost, even during peak processing loads.

Significance
The paper claims that this framework transforms SPOTLIGHT from a system limited by storage exhaustion due to false triggers into a robust platform capable of sustained, real-time transient searches. By effectively mitigating terrestrial RFI, the system enhances its sensitivity to weaker radio transients, potentially improving the detectability of faint pulsars and Fast Radio Bursts (FRBs). The work demonstrates that lightweight, real-time RFI mitigation is essential for maintaining the sensitivity and computational efficiency required by high-throughput transient surveys, validating the utility of a two-stage approach that addresses interference both before and after beamforming.

More like this