🔭 astrophysics

StarCLR: Contrastive Learning Representation for Astronomical Light Curves

StarCLR is a contrastive pretraining framework designed for large-scale astronomical light curves that learns robust temporal representations by using partially overlapping sub-sequences, demonstrating superior generalization and classification performance across diverse surveys like TESS and ZTF compared to models trained from scratch.

Original authors: Junyao Ding, Xiaodian Chen, Xinyi Gao, Xiaoyu Tang, Shu Wang, Yang Huang, Xinyu Qi, Guirong Xue, Ali Luo, Jifeng Liu

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Junyao Ding, Xiaodian Chen, Xinyi Gao, Xiaoyu Tang, Shu Wang, Yang Huang, Xinyu Qi, Guirong Xue, Ali Luo, Jifeng Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Cosmic Translator: How "StarCLR" Learns the Rhythm of the Universe

Imagine you are standing in a massive, crowded stadium during a concert. Thousands of people are making noise—clapping, singing, whistling, and cheering. If you tried to listen to every single person individually, you’d be overwhelmed. But if you were a trained musician, you could instantly pick out the beat of the drum, the melody of the singer, and the rhythm of the crowd.

In astronomy, we have a similar problem. Telescopes like TESS, ZTF, and Gaia are constantly "listening" to the sky. They record the brightness of stars over time, creating something called a light curve. These curves are like the "heartbeats" of stars. Some stars pulse like a steady drum, some flicker like a candle in the wind, and others flash like a strobe light.

The problem? We have billions of these heartbeats, and they are all recorded differently. Some telescopes take snapshots every minute; others take them every few days. It’s like trying to listen to a song where one person recorded it in high-definition studio quality, and another recorded it through a walkie-talkie in a thunderstorm.

Enter StarCLR: The Universal Rhythm Expert.

1. The "Self-Taught" Student (Contrastive Learning)

Most AI models are like students who need a teacher to show them every single flashcard: "This is a Cepheid star, this is an Eclipsing Binary." This is called "supervised learning," and it's exhausting because we don't have labels for every star in the sky.

StarCLR uses a different approach called Contrastive Learning. Imagine a student who has never seen a musical instrument before. To learn, they take a recording of a drumbeat, cut it into two slightly different pieces (maybe one piece is a bit shorter or starts a second later), and say: "I don't know what this is, but these two pieces definitely belong to the same song."

By doing this millions of times with unlabeled data from the TESS telescope, StarCLR teaches itself the "grammar" of starlight. It learns to recognize patterns, rhythms, and textures without ever being told the names of the stars.

2. The "Swiss Army Knife" of Telescopes (Generalization)

The real magic happens when we test this student. We trained StarCLR using the "clean, high-quality" music from the TESS telescope. But then, we handed it the "noisy, messy" recordings from the ZTF and Gaia telescopes.

Because StarCLR learned the fundamental essence of how light changes—rather than just memorizing specific telescope quirks—it was able to jump from one "instrument" to another with incredible accuracy. It’s like a musician who learns to play the piano and can then pick up a keyboard or an organ and play a song almost immediately.

3. The "Secret Sauce" (Hierarchical Learning & Embeddings)

The researchers added two clever tricks to make StarCLR even smarter:

The Zoom Lens (Hierarchical Loss): Instead of just looking at the whole song, the model looks at the big picture (the overall melody) and the tiny details (the individual drum hits) at the same time. This helps it catch both slow, sweeping changes and quick, sudden flashes.
The Time-Aware Compass (Positional Embedding): Since stars don't follow a perfect schedule, StarCLR uses a special mathematical "compass" that tells it exactly when each measurement happened, even if there are huge gaps in the data.

Why does this matter?

We are entering a "Golden Age" of astronomy where we are drowning in data. We can't have humans sitting at computers classifying billions of stars one by one.

StarCLR acts like an automated, highly trained conductor. It can sift through the cosmic noise, identify the different "instruments" (types of stars) playing in the dark, and help astronomers understand how the universe evolves. It turns a chaotic roar of light into a structured, understandable symphony.

Technical Summary: StarCLR: Contrastive Learning Representation for Astronomical Light Curves

1. Problem Statement

The rapid expansion of time-domain astronomical surveys (e.g., TESS, ZTF, Gaia) has generated massive volumes of light curve data. While this provides opportunities for studying stellar evolution, it poses significant challenges for variable star classification. Traditional methods rely on handcrafted features (e.g., Fourier coefficients, periods), which require extensive domain expertise and lack scalability. While deep learning offers end-to-end feature extraction, supervised models are heavily dependent on large, high-quality labeled datasets—a resource that is often scarce or survey-specific in astronomy.

2. Methodology

The authors propose StarCLR, a self-supervised contrastive pretraining framework designed to learn robust temporal representations from large-scale, unlabeled astronomical light curves.

Core Architecture: StarCLR utilizes an encoder-only Transformer backbone. To handle the inherent irregularity of astronomical observations, the model employs a time-aware sinusoidal positional encoding. Instead of using discrete integer indices, it uses continuous observation times scaled by $2\pi$ , allowing the model to capture phase-like periodic components across various timescales.
Data Augmentation & Contrastive Objective: To facilitate self-supervised learning, the model uses a "duplication + random cropping" strategy. It creates positive pairs by extracting two partially overlapping subsequences from a single light curve.
Hierarchical Contrastive Loss: The framework employs a dual-level loss function:
1. Instance-wise Contrastive Loss: Ensures that different augmented views of the same light curve are close in representation space while pushing different instances apart.
2. Temporal Contrastive Loss: Enhances sensitivity to intra-source temporal dynamics by comparing features across different time steps within the same instance.
3. Hierarchical Structure: A max-pooling operation is applied at multiple layers to capture multi-scale temporal features (both local and global).
Fine-tuning Strategy: After pretraining on TESS data, the backbone is frozen, and a lightweight multi-layer perceptron (MLP) classification head is added. To improve performance, the model concatenates the learned temporal embeddings with supplementary astrophysical features (e.g., period, amplitude, parallax, color index).

3. Key Contributions

Survey-Agnostic Representations: Unlike previous models that are often survey-specific, StarCLR is pretrained on one survey (TESS) and successfully transfers to others (ZTF, Gaia) with different cadences and sampling patterns.
Handling Irregular Sampling: By integrating continuous time into the positional encoding and using a Transformer architecture, the model effectively manages the irregular gaps and varying cadences typical of astronomical data.
Hierarchical Learning: The introduction of a hierarchical contrastive loss allows the model to capture both fine-grained local variations and broad global periodicities.
Feature Integration: The framework demonstrates a successful hybrid approach, combining deep learned morphological features with traditional physical astrophysical parameters.

4. Results

The model was evaluated on three major surveys with distinct observational characteristics:

TESS (High cadence, 18 classes): Achieved a macro-F1 of 84.35% and micro-F1 of 94.46%.
ZTF (Ground-based, sparse sampling, 11 classes): Achieved a macro-F1 of 87.82% and micro-F1 of 92.83%. Notably, StarCLR showed its largest gains over baseline models on this sparsely sampled dataset.
Gaia (Scanning law, sparse, 24 classes): Achieved a macro-F1 of 92.73% and micro-F1 of 99.49%. While performance was high, the authors noted that Gaia's results are heavily driven by the inclusion of physical astrophysical features.

Ablation Studies revealed:

Architecture: Transformers significantly outperformed LSTMs in capturing long-range dependencies.
Embedding: Time-aware positional encoding with $2\pi$ scaling was superior to simple joint time-flux embeddings.
Pretraining Data: Pretraining on dense data (TESS) favors dense downstream tasks, while pretraining on sparse data (ZTF) improves performance on sparse downstream tasks, suggesting a "domain alignment" effect.

5. Significance

StarCLR represents a significant step toward foundation models for time-domain astronomy. By demonstrating that a model can learn universal temporal "grammar" from unlabeled data, it reduces the reliance on expensive manual labeling. Its ability to generalize across different telescope systems (space-based vs. ground-based) makes it a highly scalable tool for the next generation of large-scale astronomical surveys, providing a robust backbone for automated stellar classification and anomaly detection.