Pre-training vision models for the classification of alerts from wide-field time-domain surveys

This paper demonstrates that adopting standardized computer vision architectures pre-trained on astronomical data, particularly from Galaxy Zoo, significantly improves the performance and efficiency of alert classification in wide-field time-domain surveys compared to traditional custom CNNs trained from scratch.

The Gist

Imagine you are the manager of a massive, 24/7 security camera system that watches the entire night sky. Every few seconds, this system spots something new: a star that flickered, a galaxy that moved, or a rock flying through space. These are called "alerts."

The problem? The system is so sensitive that it generates millions of alerts every night. Most of them are just glitches, dust on the lens, or boring old stars. Only a tiny fraction are the "golden tickets"—rare, exciting cosmic events like exploding stars or asteroids on a collision course with Earth.

Your job is to find the golden tickets before they disappear. But you can't look at every single photo yourself; you'd go crazy. So, you need a robot assistant (an AI) to do the filtering for you.

This paper is about teaching that robot assistant how to be the best possible detective.

The Old Way: Hiring a Fresh Graduate

For years, astronomers built their own custom AI robots from scratch. It was like hiring a bright college graduate and saying, "Here is a stack of 10,000 photos of space glitches and explosions. Figure out the rules yourself."

The robot would study the photos, make mistakes, learn, and eventually get good at the job. But it took a long time to train, and the robot was only good at this specific job. If you wanted it to look at a different type of photo, you'd have to start over.

The New Way: Hiring a Seasoned Expert

The authors of this paper asked: "What if we didn't start from scratch?"

In the world of regular computer vision (like your phone's face unlock or self-driving cars), there's a standard practice: Pre-training.

• The Analogy: Instead of hiring a fresh graduate, you hire a seasoned detective who has already spent 10 years solving thousands of different cases (like identifying cats, cars, and traffic signs in everyday photos).
• The Process: You take this expert, give them a short "refresher course" on space photos, and they immediately become a top-tier space detective.

The paper tested two different "refresher courses" (pre-training datasets):

1. ImageNet: The standard dataset of everyday objects (cats, dogs, cars).
2. Galaxy Zoo: A dataset where real humans classified actual pictures of galaxies.

The Surprising Results

The researchers ran a massive experiment, testing different AI "brains" (architectures) and different training methods. Here is what they found:

1. The "Galaxy Zoo" Expert Wins
You might think an expert trained on everyday objects (ImageNet) would be the best starting point. But the paper found that the expert trained on Galaxy Zoo (real space images) was the clear winner.

• Why? Even though the Galaxy Zoo photos looked different from the alert photos, the expert had already learned how to recognize the "shape" of a galaxy, the texture of a star, and the noise of a telescope. It was like teaching a detective who knows how to spot a fingerprint to look for a specific type of shoe print. The underlying skills transferred perfectly.

2. The "Off-the-Shelf" Models are Faster and Smarter
The researchers also tested two modern, pre-made AI architectures (ConvNeXt and MaxViT) against their old custom model.

• The Result: The modern models were not only more accurate, but they were also much faster and required less computer memory to run.
• The Analogy: The old custom model was like a heavy, clunky truck that could carry a load but moved slowly. The new pre-trained models were like high-speed electric sports cars: they could carry the same (or more) load, but they zipped through the data with ease. This is crucial because the new telescopes (like the upcoming LSST) will generate so many alerts that slow computers will cause a traffic jam, causing us to miss the golden tickets.

3. Less Data Needed
The most exciting finding is that these pre-trained models didn't need as many examples to learn.

• The Analogy: If you give a fresh graduate 100 photos to learn from, they might struggle. But if you give a seasoned expert just 10 photos, they can figure it out instantly because they already know the basics. This is vital because labeling space data is hard and expensive; we don't have millions of labeled examples for every new type of cosmic event.

The Big Picture

The authors are essentially saying: "Stop reinventing the wheel."

For the next generation of space telescopes, which will flood us with data, we need to stop building custom AI robots from scratch. Instead, we should take the best, most efficient AI models that already exist, give them a quick "space refresher" using real galaxy images, and let them do the heavy lifting.

This approach will make our space surveillance faster, cheaper, and more accurate, ensuring we catch every exploding star and incoming asteroid without getting bogged down by computer traffic jams.

In short: The best way to teach a robot to see the universe is to first teach it to see the world, and then show it a few pictures of galaxies. It works better, faster, and with less effort.

Technical Summary

Here is a detailed technical summary of the paper "Pre-training vision models for the classification of alerts from wide-field time-domain surveys."

1. Problem Statement

Modern wide-field time-domain surveys (e.g., ZTF, ATLAS, and the upcoming LSST) generate massive volumes of "alerts" via image differencing. These alerts require rapid filtering to identify scientifically valuable transients (e.g., supernovae, variable stars) while rejecting artifacts ("bogus" alerts) and less interesting sources.

Currently, the astronomy community predominantly relies on custom Convolutional Neural Networks (CNNs) trained from scratch for this task. This approach faces several challenges:

• Data Scarcity: Labeling astronomical data is expensive and slow, often resulting in small training sets compared to the data volume generated.
• Inefficiency: Designing and tuning custom architectures is time-consuming and often yields suboptimal computational efficiency.
• Paradigm Lag: While the broader computer vision (CV) field has shifted toward using standardized, pre-trained architectures (e.g., on ImageNet) to leverage transfer learning, astronomy has largely remained stuck in the "train from scratch" paradigm.

The paper investigates whether adopting modern CV practices—specifically pre-training and off-the-shelf architectures—can improve the performance and efficiency of alert classification in astronomy.

2. Methodology

The authors conducted a comprehensive experimental design comparing various model configurations on a bright transient candidate vetting task (distinguishing real extragalactic transients from other sources).

A. Model Architectures

They compared three distinct architectural approaches:

1. Baseline Custom CNN: A small, specialized CNN typical of existing astronomical literature (similar to the production BTSbot model).
2. ConvNeXt (Pico): A modern, fully-convolutional architecture that reinterprets ResNet with transformer-inspired design choices (large kernels, layer normalization).
3. MaxViT (Tiny): A Vision Transformer (ViT) variant utilizing "multi-axis" attention mechanisms to capture both local and global relationships.

B. Pre-training Regimens

Models were evaluated under three training conditions:

1. From Scratch: Random initialization of weights.
2. ImageNet Pre-training: Standard pre-training on ~1.28 million everyday object images (non-astronomical).
3. Galaxy Zoo Pre-training: Pre-training on ~842,000 annotated galaxy images from the Galaxy Zoo citizen science project (astronomical domain).

C. Datasets and Tasks

• Primary Task: Binary classification of ZTF (Zwicky Transient Facility) alert cutouts (Science, Reference, Difference images).
• Secondary Task: Multi-modal classification (combining image cutouts with metadata) to simulate production workflows for triggering spectroscopic follow-up.
• Fine-tuning Data: The authors used the BTSbot dataset (high-quality labels for bright transients) and created subsets of varying sizes (5%, 10%, 50%, 100% of ~769k alerts) to test data efficiency.
• Distribution Shift Analysis: They also fine-tuned models on DESI Legacy Survey color images to test performance when the fine-tuning data distribution (RGB) matches the pre-training data (RGB) more closely than the ZTF alert cutouts (Science-Ref-Diff).

D. Evaluation Metrics

• Classification Quality: ROC AUC (Area Under the Receiver Operating Characteristic Curve).
• Production Utility: F1 Score of triggers (harmonic mean of purity and completeness) for automated follow-up allocation.
• Efficiency: Inference throughput (alerts/second) on GPU and CPU, and memory usage.
• Qualitative Analysis: UMAP embeddings to visualize the latent feature spaces and identify outliers/mislabeling.

3. Key Contributions

1. Paradigm Shift Validation: The paper provides empirical evidence that the astronomy community should shift from training custom CNNs from scratch to adopting pre-trained, off-the-shelf architectures.
2. Domain-Specific Pre-training Superiority: It demonstrates that pre-training on Galaxy Zoo (astronomical data) consistently outperforms pre-training on ImageNet (everyday objects) and training from scratch, even when the downstream task involves different image modalities (difference images vs. RGB).
3. Efficiency vs. Performance Trade-off: The study highlights that modern architectures (specifically ConvNeXt) offer superior computational efficiency (higher throughput, lower memory) compared to custom baselines, despite having more parameters.
4. Data Efficiency: Pre-trained models achieve high performance with significantly fewer labeled examples, a critical advantage for surveys like LSST where labeling cannot keep pace with data generation.
5. Open Source Release: The authors released the pre-trained models and training code on Hugging Face and GitHub, facilitating immediate adoption by the community.

4. Key Results

• Performance:
  • Galaxy Zoo Pre-training yielded the best results. The MaxViT model pre-trained on Galaxy Zoo achieved the highest ROC AUC (approx. 0.025 higher than the baseline custom CNN).
  • ConvNeXt pre-trained on Galaxy Zoo matched or slightly exceeded the baseline custom CNN across all training set sizes.
  • ImageNet Pre-training provided a modest boost over training from scratch but generally underperformed compared to Galaxy Zoo pre-training.
  • Training from Scratch: Custom CNNs and modern architectures trained from scratch generally underperformed or matched the baseline, failing to leverage the full potential of the data.
• Multi-modal Performance:
  • For small training sets (<100k alerts), the baseline custom multi-modal CNN outperformed off-the-shelf models.
  • For larger training sets (>300k alerts), off-the-shelf architectures (ConvNeXt/MaxViT) surpassed the baseline, likely due to higher learning capacity.
• Computational Efficiency:
  • ConvNeXt was the most efficient: 6.4x faster on CPU and 5.5x lower memory usage than the custom CNN, despite having more trainable parameters.
  • MaxViT was the least efficient (20x slower on CPU, 16x more memory) due to the computational cost of attention mechanisms, though it offered the highest peak performance.
• Latent Space & Outlier Detection:
  • UMAP analysis showed that pre-trained models learned similar latent structures to the baseline but with distinct substructures.
  • Combining UMAP embeddings with Isolation Forests successfully identified rare astrophysical phenomena (e.g., twin Type Ia supernovae) and label noise in the training set (e.g., duplicate sources mislabeled as rejects).

5. Significance and Conclusion

This paper advocates for a paradigm shift in time-domain astronomy. The authors conclude that:

• Adopting Pre-training: Astronomers should abandon "train from scratch" approaches in favor of pre-training on relevant astronomical datasets (like Galaxy Zoo) or even general datasets (ImageNet) if astronomical data is unavailable.
• Architecture Choice: ConvNeXt is recommended as the optimal balance of performance and computational efficiency for real-time alert processing, particularly for CPU-constrained environments common in survey brokers. MaxViT is suitable for high-priority, ambiguous alerts where maximum accuracy is required and computational cost is secondary.
• Future Readiness: As the field transitions to the LSST era (generating ~100 alerts per second), the efficiency gains of off-the-shelf models are critical to prevent alert processing bottlenecks.
• Production Deployment: The authors have already deployed a multi-modal ConvNeXt model (pre-trained on Galaxy Zoo) into the production BTSbot workflow, replacing the legacy custom CNN to handle the increased data load of upcoming surveys.

In summary, the study proves that leveraging the latest computer vision practices yields models that are more accurate, more data-efficient, and significantly faster than the custom solutions currently standard in the field.