Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector

Technical Summary: Generalist Vision-Language Models for Fast Radio Burst Detection

Problem Statement
The automated detection of Fast Radio Bursts (FRBs)—millisecond-duration, extragalactic radio transients—increasingly relies on specialized deep learning models. While these detectors (e.g., SwinYNet) achieve exceptional performance, they require large, task-specific training datasets and cannot be redefined without retraining. Furthermore, traditional pipelines suffer from high computational costs due to brute-force dedispersion and generate significant false positives, particularly from structured Radio Frequency Interference (RFI), necessitating human verification. This paper investigates whether small, open-weight, locally run generalist Vision-Language Models (VLMs) can detect FRBs in dynamic spectra under a zero-shot, prompt-only regime. The core question is whether a generalist foundation model, with no FRB-specific training or fine-tuning, can recognize the visual signatures of dispersed bursts and distinguish them from RFI and noise.

Methodology
The study employs a controlled, simulated benchmark to ensure reproducibility and label control.

• Dataset: Using simulateSearch, the authors generated 3,000 L-band dynamic spectra (2 seconds each, 2048 frequency channels). The set includes 1,000 FRB examples (with varying Dispersion Measures, widths, and flux densities), 1,000 structured RFI examples (five subtypes: persistent narrowband, impulsive broadband, wifi multiband, point-to-point microwave, and satellite-like inband), and 1,000 pure noise samples.
• Benchmark Construction: A balanced binary subset of 2,000 samples (1,000 FRB, 500 RFI, 500 Noise) was used for the primary evaluation. A full 3,000-sample set was used for a secondary multiclass task.
• Models: Two small, open-weight generalist VLMs from the Gemma 4 family were evaluated: Gemma 4 2B and Gemma 4 4B. Both were run locally on a workstation with NVIDIA RTX 4500 Ada GPUs.
• Protocol: The models operated in a strict zero-shot, prompt-only regime. No weights were updated, and no labeled examples were provided in the prompt.
  • Input: The VLMs received only anonymized PNG images of the dynamic spectra (visualizing time vs. frequency with signal power) and a natural language instruction defining the classes. They did not receive the native PSRFITS files, simulation parameters, or metadata.
  • Baseline: The VLMs were compared against SwinYNet, a state-of-the-art specialized detector trained on FRB data, which operates directly on the native PSRFITS files.
  • Output: The VLMs were prompted to return a structured JSON object containing a binary label (FRB/NON-FRB), a continuous frb_probability score, confidence, and a natural language justification.
• Evaluation: Metrics included accuracy, macro-F1, precision, recall, ROC-AUC, Average Precision (AP), and the Brier score. A paired McNemar test was used to assess statistical significance between models on the same samples.

Key Results

1. Binary Classification Performance: At the default threshold of 0.5, Gemma 4 2B achieved an accuracy of 93.65% and a macro-F1 of 0.9364. This was statistically indistinguishable from the specialized SwinYNet detector (92.90% accuracy, macro-F1 0.9286), despite the VLM having no domain-specific training. Gemma 4 4B achieved 90.45% accuracy.
2. False Positive Profiles: The VLMs demonstrated a distinct advantage in rejecting structured interference.
   • RFI Rejection: Gemma 4 2B had a false-positive rate (FPR) of 6.4% on RFI, compared to 25.0% for SwinYNet. Gemma 4 4B was even more conservative with an RFI FPR of 3.0%.
   • Noise Rejection: Neither VLM produced any false positives on pure noise samples, whereas SwinYNet had an FPR of 3.4%.
   • Trade-off: The VLMs were more conservative than SwinYNet, resulting in lower recall for FRBs (Gemma 4 2B: 90.5% vs. SwinYNet: 100%). The VLMs tended to miss faint FRBs, classifying them as noise.
3. Probabilistic Ranking: SwinYNet achieved a perfect ROC-AUC of 1.0000, indicating perfect ranking of FRBs above non-FRBs. The VLMs approached this ceiling with ROC-AUCs of 0.9482 (Gemma 4 2B) and 0.9309 (Gemma 4 4B). However, the VLM scores were heavily quantized (Gemma 4 2B used only three distinct probability values), limiting the granularity of their probabilistic calibration.
4. Multiclass Capability: By simply rewriting the prompt (without retraining), the models were reconfigured for a three-class task (FRB/RFI/Noise). On the full 3,000-sample set, Gemma 4 4B reached 86% accuracy with zero false FRB classifications (perfect precision for the FRB class), demonstrating the ability to separate pure noise from structured RFI.

Significance and Claims
The paper claims that generalist VLMs possess a latent capacity to recognize the visual patterns of dispersed radio bursts without any domain-specific training. The significance of the work lies in demonstrating that:

• Generalist Transfer: A general-purpose model can approach the performance of a specialized detector on a scientific task using only natural language prompts and visual input.
• Complementary Error Profiles: While specialized detectors maximize recall, generalist VLMs may offer superior rejection of structured RFI, potentially reducing the human inspection load in real-world pipelines.
• Operational Flexibility: The ability to redefine the classification task (e.g., from binary to multiclass) solely through prompt engineering offers a flexibility that specialized detectors lack without retraining.
• Interpretability: Unlike specialized detectors, VLMs provide natural language justifications for their decisions, offering an auditable layer of explainability.

The authors explicitly state that this work does not claim the VLMs are superior to specialized detectors (which maintain a perfect probabilistic ceiling) nor that they are ready for brute-force real-time search (due to inference latency of ~5–8 seconds per file). Instead, the results serve as a "conservative floor" for the potential of generalist models, suggesting they are viable for candidate triage, hybrid pipelines, and scenarios where interpretability and adaptability are prioritized. The paper concludes that while validation on real observational data is necessary, the zero-shot approach represents a promising direction for reducing the computational and human bottlenecks in FRB detection.