🔭 astrophysics

Enhanced detection limits in the SHINE F150 survey through the Regime Switching Model. Optimizing thresholds and investigating environmental noise

This study enhances detection limits in the SHINE F150 survey by applying the Regime Switching Model to 213 VLT/SPHERE observations, utilizing environmental clustering and optimized thresholding to achieve a two- to five-fold improvement over standard PCA processing and identify over 30 new signals.

Original authors: Mariam Sabalbal, Olivier Absil, Carl-Henrik Dahlqvist, Philippe Delorme

Published 2026-02-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mariam Sabalbal, Olivier Absil, Carl-Henrik Dahlqvist, Philippe Delorme

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to take a photograph of a tiny, glowing firefly hovering just next to a blindingly bright spotlight. The spotlight is so intense that it creates a "glare" and "flickering dust motes" (called speckles) all around it, making it nearly impossible to see the firefly. This is exactly the challenge astronomers face when trying to find exoplanets (the fireflies) orbiting distant stars (the spotlights).

This paper is about a team of astronomers who built a smarter, more sophisticated "camera filter" to help them see those fireflies more clearly. Here is the story of how they did it, broken down into simple concepts.

1. The Problem: The "Glare" is Too Loud

The team used a powerful telescope called VLT/SPHERE to look at 150 young stars. They had a lot of data (photos), but the "noise" from the star's glare was messy. Sometimes the wind outside the telescope would shake the air, creating weird patterns. Other times, the air was calm, but the telescope itself had a glitch called the "low-wind effect."

Previously, astronomers used a standard method (like a basic photo editor) to try to remove the glare. It worked okay, but it missed many faint planets, especially those close to the star.

2. The Solution: The "Regime Switching Model" (RSM)

The authors introduced a new algorithm called the Regime Switching Model (RSM). Think of this not as a simple filter, but as a super-smart detective.

Old Method: The old way looked at the photo and said, "That bright spot looks like noise, so I'll delete it."
The RSM Detective: This detective looks at the story of the pixels over time. It knows that noise (the glare) behaves chaotically, like static on a TV. But a real planet moves in a predictable, smooth path, like a dancer.
The Trick: The RSM combines several different "photo editing" techniques at once. It's like asking three different art critics to look at the same painting and agreeing on what is real. If all three say, "That's a planet," the detective is confident.

3. Grouping the Data: The "Weather Report"

The team realized that not all photos are the same. A photo taken on a windy, stormy night looks different from one taken on a calm, clear night.

To handle this, they used a computer trick called Clustering. Imagine you are sorting a huge pile of laundry. Instead of mixing everything together, you sort it by "weather conditions":

Group A: Windy, shaky nights (creates a "wind-driven halo" of noise).
Group B: Calm, clear nights.
Group C: Nights with a specific telescope glitch.

By sorting the data into these groups, they could tune their "detective" to be perfect for each specific type of weather. They found that the noise in a windy group was totally different from the noise in a calm group. If you treat them all the same, you either miss planets or see things that aren't there.

4. Setting the Rules: The "Security Guard"

Once the detective has processed the photos, it needs a rule to decide: "Is this a real planet or just a glitch?"

The team tested two ways to set this rule:

The "Statistical Guard" (Lognormal): This guard says, "If a signal is brighter than 99.9999% of the random noise we've seen before, it's a planet." This is very safe and conservative.
The "Balanced Guard" (F1 Score): This guard tries to find the perfect balance between catching every real planet and not catching any fake ones. It's like a bouncer at a club trying to let in all the VIPs but no impostors.

They found that both guards worked well, but the "Statistical Guard" was better for handling a huge survey like this one because it was consistent.

5. The Results: Seeing the Invisible

When they applied this new, smarter detective to the 150 stars, the results were amazing:

Double the Vision: At medium distances from the star, they could see planets twice as faint as before.
Five Times Better: Very close to the star (where the glare is worst), they could see planets five times fainter than before.
New Discoveries: They found 38 new signals that the old method missed.
- Most turned out to be background stars (stars behind the target star).
- Some were false alarms (glitches).
- One looks like a brand-new, promising exoplanet candidate that needs a second look to confirm.

The Big Picture

Think of this paper as upgrading from a standard pair of sunglasses to a high-tech, AI-powered visor. By understanding that the "weather" (atmospheric conditions) changes the nature of the glare, and by using a detective that looks at the story of the light rather than just a single snapshot, the team has significantly improved our ability to find new worlds.

They didn't just find one new planet; they built a better system that will help astronomers find many more in the future, bridging the gap between what we can see with our eyes and what is hiding in the shadows of the universe.

1. Problem Statement

High-contrast imaging (HCI) aims to directly detect faint exoplanets near bright host stars. While instruments like VLT/SPHERE have made significant strides, detection limits are often constrained by residual stellar speckles and noise that do not follow simple Gaussian distributions.

Limitations of Current Methods: Standard post-processing techniques (e.g., PCA, TLOCI) often struggle at small angular separations (inner working angles). Furthermore, detection thresholds are frequently set using a fixed $5\sigma$ assumption based on Gaussian noise, which may not accurately reflect the complex, non-Gaussian noise characteristics of HCI data, particularly under varying environmental conditions (e.g., wind-driven halos, low-wind effects).
The Gap: There is a need for advanced statistical tools that can better model the temporal and spatial evolution of noise, adapt to specific observational environments, and provide robust, data-driven detection thresholds to identify fainter companions.

2. Methodology

The authors applied the Regime Switching Model (RSM), a statistical algorithm adapted from econometrics, to the F150 sample of the SHINE survey (150 stars, 213 observations). The methodology involved four key pillars:

A. Environmental Clustering

To account for varying noise characteristics, the authors grouped the 213 observations into six distinct clusters using k-means clustering. The clustering was based on environmental and instrumental parameters:

Seeing, wind speed, coherence time, Strehl ratio, raw contrast, number of frames, and parallactic angle range.
Purpose: This allowed the authors to identify specific noise regimes (e.g., Cluster 1 associated with low-wind effects; Cluster 5 with wind-driven halos) and optimize algorithm parameters for representative datasets within each cluster rather than treating all data uniformly.

B. The RSM Algorithm Application

The RSM algorithm was applied to the residual, de-rotated frames after PSF subtraction.

Mechanism: It models pixel intensity time series using a Markov regime-switching framework with two states: a noise regime and a planetary regime.
Integration: RSM combines outputs from multiple PSF subtraction techniques (APCA, NMF, LOCI, FM-KLIP, FM-LOCI) to construct a unified time series, leveraging temporal noise diversity to distinguish true signals from speckles.
Optimization: Parameters were optimized for cluster centers and generalized to all members of the cluster to balance computational efficiency with performance.

C. Detection Threshold Strategies

The study proposed and compared two methods for defining radial detection thresholds:

Lognormal Distribution Fitting: Fitting a lognormal distribution to the noise in the final RSM maps (excluding bright signals) and setting a threshold at a False Alarm Probability (FAP) of $3 \times 10^{-7}$ (equivalent to $5\sigma$ in Gaussian statistics). This method is computationally efficient and conservative.
F1 Score Maximization: Empirically maximizing the F1 score (balancing True Positives and False Positives) via fake companion injections. This method is dataset-specific but computationally expensive.

Decision: For the large survey, the Lognormal approach was adopted for its robustness and generalizability, while the F1 score was validated as a complementary, observation-specific metric.

D. Contrast Curve Generation

The authors generated 50% completeness contrast curves by injecting fake companions and assessing recovery rates. They evaluated various combinations of PSF subtraction techniques to determine the optimal configuration for RSM.

3. Key Contributions

Environmental Clustering: Demonstrated that clustering observations by environmental parameters (wind, seeing, etc.) is crucial for understanding noise behavior and optimizing detection limits.
Threshold Optimization: Validated that detection thresholds vary significantly (by up to a factor of 10) across different environmental clusters, arguing against a "one-size-fits-all" threshold.
RSM Configuration: Identified that combining multiple PSF subtraction techniques (specifically APCA-NMF-LOCI-FMKLIP and APCA-LOCI-FMKLIP) yields the narrowest noise distributions and best contrast.
Threshold Comparison: Showed that Lognormal and F1-score-based thresholds are generally compatible, with Lognormal offering a conservative, survey-wide standard.

4. Results

Detection Limit Improvements:
- RSM improved detection limits by a factor of ~2 at 1 arcsecond and ~5 at inner working angles (135 mas) compared to standard full-frame PCA processing.
- This represents a magnitude improvement of 1.8 mag at 0.135" over previous PCA results.
- The performance is comparable to the PACO algorithm (Chomez et al. 2023), confirming RSM as a state-of-the-art tool.
Noise Characterization:
- The study confirmed that noise distributions are lognormal and highly dependent on observing conditions.
- Wind-driven halos (Cluster 5) and low-wind effects (Cluster 1) were successfully isolated, showing distinct noise tails and threshold requirements.
Signal Detection:
- Total Signals: 87 signals were identified across 59 observations.
- Recovery: 49 signals matched previous PACO detections.
- New Detections: 38 new signals were found that PACO missed.
  - Classification: Of the new signals, 22 were identified as background stars, 14 as false positives, 1 as a known low-mass companion, and 1 as a promising new exoplanet candidate (awaiting second-epoch confirmation).
- Specific Success: RSM successfully recovered PZ Tel in a 2014 epoch where PACO failed, and identified a background star around HD 164249 via proper motion tests.

5. Significance

Statistical Rigor: The paper establishes a robust framework for setting detection thresholds in HCI that accounts for non-Gaussian noise and environmental variability, moving beyond simple Gaussian assumptions.
Survey Efficiency: By clustering data, the authors demonstrated a scalable method to optimize post-processing parameters for large surveys without the prohibitive cost of individual dataset optimization.
Scientific Impact: The application of RSM to the SHINE F150 sample significantly deepens the survey's sensitivity, particularly at small separations where the majority of giant exoplanets are expected to form. The identification of a new candidate and the recovery of missed signals highlight the algorithm's potential to refine exoplanet demographics and formation theories.
Generalizability: The methodology is designed to be transferable to other ground-based high-contrast imaging surveys (e.g., GPIES, BEAST, ISPY), providing a standardized approach for future exoplanet detection efforts.