🔭 astrophysics

Tabular foundation models for the estimation of probabilistic quasar photometric redshifts in S-PLUS

This paper demonstrates that the tabular foundation model TabPFN 2.5 serves as a robust, off-the-shelf solution for estimating probabilistic quasar photometric redshifts in the S-PLUS survey, outperforming various task-specific baselines particularly in scenarios with limited training data, extreme source brightness, or significant covariate shifts, despite requiring substantial computational resources for inference.

Original authors: Raquel R. Valença, Lilianne Nakazono, Rafael Izbicki, Marco Henrique de Almeida Inácio, Bruno Marcondes e Resende, Maycon Jorge Deláqua da Silva, Kiana Coimbra Buin Lins, Natanael M. Cardoso, Claudia

Published 2026-08-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Raquel R. Valença, Lilianne Nakazono, Rafael Izbicki, Marco Henrique de Almeida Inácio, Bruno Marcondes e Resende, Maycon Jorge Deláqua da Silva, Kiana Coimbra Buin Lins, Natanael M. Cardoso, Claudia Mendes de Oliveira

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the universe as a giant, cosmic library where every star and galaxy is a book. To understand the story of the universe—how it grew, how old it is, and how the stars inside it behave—we need to know exactly where each book is located in space and time. In astronomy, this location is called "redshift." It's a bit like a cosmic barcode: the faster a galaxy is moving away from us, the more its light stretches out, shifting toward the red end of the rainbow. By measuring this shift, astronomers can tell how far away an object is and how long ago its light began its journey.

However, getting this "barcode" perfectly is tricky. The most accurate way to read it is to take a high-resolution "spectroscopic" photo, which breaks the light down into a detailed rainbow. But this process is like hiring a master librarian to read every single book one by one; it takes forever and is incredibly expensive. So, for the billions of objects in modern sky surveys, astronomers have to guess the redshift based on a simpler, "photometric" photo—just a few snapshots of the object in different colored filters. The problem is that this guess is often like trying to guess a person's age just by looking at a blurry, low-resolution photo; different ages can look surprisingly similar, leading to confusing, multi-possible answers.

This is where a new kind of AI comes in to help. Scientists are testing whether "foundation models"—super-smart, pre-trained AI systems that have already learned the rules of data from millions of other examples—can act as instant, off-the-shelf experts for guessing these cosmic distances. The big question is: Can these general AI tools, which haven't been specifically taught about quasars (the super-bright cores of distant galaxies), handle the messy, confusing reality of real-world sky data better than the specialized tools astronomers have built over the last decade?

In this paper, the authors put this idea to the test using data from the S-PLUS survey, a massive project mapping the southern sky with 12 different color filters. They focused on quasars, which are particularly tricky because their colors can be deceivingly similar at very different distances, creating a "color-redshift degeneracy" where one set of colors could mean several different distances. The team compared three new "tabular foundation models" (specifically TabPFN 2.5, RealTabPFN 2.5, and TabICL) against eight traditional, specialized machine-learning methods.

Think of the traditional methods as specialized apprentices who have spent years studying only quasars. The foundation models, on the other hand, are like generalist geniuses who have read every book in the library but haven't specifically studied quasars yet. The researchers wanted to see if these generalists could step in, look at the data, and instantly produce a full "probability map" of where a quasar might be, rather than just guessing a single number. This is crucial because, as the paper notes, a single guess is often wrong; a probability map tells you, "It's probably here, but there's a small chance it's way over there," which helps avoid catastrophic errors in future cosmic studies.

The results were surprisingly clear. The foundation models, particularly TabPFN 2.5, performed exceptionally well, often beating or tying with the best specialized tools. The most impressive part was that these generalist models shined brightest when the data was scarce. When the team gave them only 500 quasars to learn from (a tiny amount in astronomy terms), the foundation models were far superior to the specialized tools, which struggled to learn from such a small sample. Even with a massive dataset of over 121,000 quasars, TabPFN 2.5 remained a top contender, producing highly accurate probability maps that were well-calibrated, meaning their "guesses" about uncertainty were honest and reliable.

The study also tackled a sneaky problem called "covariate shift." Imagine you trained a weather forecaster using data only from sunny days, but then asked them to predict rain. They might fail because the conditions are different. Similarly, the quasars astronomers can easily study (the "training set") are often brighter and easier to see than the billions of faint quasars they actually want to study (the "target population"). The paper used a clever weighting technique to simulate how the models would perform on these faint, difficult-to-see objects. Even under this stress test, TabPFN 2.5 held its ground, staying stable while some of the specialized tools became overconfident and made more mistakes.

By analyzing which features the AI used to make its guesses, the authors found that the models relied heavily on data from the WISE infrared telescope (specifically the W1 and W2 bands) and the GALEX ultraviolet telescope, in addition to the optical colors from S-PLUS. It's as if the AI realized that to see the true distance of these cosmic beacons, you need to look at them not just in visible light, but also in infrared and ultraviolet.

However, the paper does point out a practical catch. While these models are powerful, they are computationally hungry. Using the full dataset of over 120,000 training quasars to make predictions requires significant computer memory and time. The authors suggest that for future massive surveys, they might need to "distill" these models or use smaller subsets of data to make the process faster.

In conclusion, the paper suggests that TabPFN 2.5 is a strong, reliable "default" choice for estimating quasar distances, especially when data is limited or when getting the uncertainty right is critical. It proves that you don't always need to build a custom tool from scratch; sometimes, a pre-trained, generalist AI can step in and do the job just as well, if not better, than the experts. This opens the door for using these powerful foundation models in future astronomical surveys to help us map the universe with greater precision and fewer errors.

Technical Summary: Tabular Foundation Models for Probabilistic Quasar Photometric Redshifts in S-PLUS

Problem Statement
Estimating photometric redshifts ( $z_{photo}$ ) for quasars in wide-field surveys like the Southern Photometric Local Universe Survey (S-PLUS) is critical for cosmological studies but faces two primary challenges. First, the mapping from observed photometric features to redshift is inherently multi-valued due to color-redshift degeneracies, broad emission lines, and dust reddening; thus, a single point estimate is often inadequate, necessitating the estimation of the full conditional density $p(z|x)$ . Second, a significant covariate shift exists between the spectroscopic training sample (which is biased by selection functions, magnitude limits, and follow-up priorities) and the target photometric population. Standard evaluation on spectroscopic test sets may therefore misrepresent performance on the actual survey catalog. While task-specific machine learning models (e.g., random forests, mixture density networks) have been the standard, the emergence of tabular foundation models offers a new class of "off-the-shelf" tools that require no survey-specific training.

Methodology
The authors benchmark three tabular foundation models—TabPFN 2.5, RealTabPFN 2.5, and TabICL—against eight established baselines for quasar photo-z estimation using S-PLUS Data Release 6 (DR6). The dataset comprises 12-band optical photometry (S-PLUS), complemented by UV (GALEX) and mid-infrared (WISE) data, resulting in a 39-dimensional feature vector for 135,164 spectroscopically confirmed quasars.

The benchmark compares:

Foundation Models: Frozen, pre-trained transformers (TabPFN 2.5, RealTabPFN 2.5, TabICL) that use the spectroscopic training set as "in-context support" for inference without weight updates.
Task-Specific Baselines: Linear conditional Gaussians, FlexZBoost, Mixture Density Networks (MDN, MDN-deep), Normalizing Flows (Flow-Spline), Random Forests (RF-Point), and Gradient Boosted Machines (GBM-Point).

The evaluation employs a dual-regime approach:

Standard Evaluation: Metrics computed on the spectroscopic test set.
Importance-Weighted Evaluation: Metrics reweighted to approximate the photometric target population, addressing covariate shift. Weights are estimated via a classification task (distinguishing spectroscopic from photometric objects) and tempered with an exponent $\alpha$ to control variance (specifically $\alpha \approx 0.36$ for a 30% effective sample size and $\alpha=1$ for the raw target).

Performance is assessed using density metrics (CDE loss, log-likelihood, CRPS, PIT KS statistic, 90% coverage) and point-prediction metrics (RMSE, Bias, NMAD, outlier fractions). Scaling is tested across training set sizes from $n=500$ to $n=121,626$ .

Key Results

Overall Performance: TabPFN 2.5 emerges as the most robust method, achieving the best or statistically tied-for-best performance across nearly all density and point-prediction metrics. The only exception is the unweighted CDE loss, where the normalizing flow (Flow-Spline) is statistically tied and attains the lowest mean value, likely due to direct optimization of that specific loss function.
Covariate Shift Robustness: Under importance-weighted evaluation (simulating deployment on the photometric target), TabPFN 2.5 maintains superior calibration and lower catastrophic outlier rates compared to baselines. Methods like FlexZBoost show significant degradation in coverage and calibration under shift, whereas TabPFN 2.5 remains stable.
Data Regimes: The advantage of foundation models is most pronounced in the small-data regime ( $n_{train} \le 1,000$ ), where they significantly outperform baselines due to strong inductive biases from pre-training. At the full DR6 training size, they remain competitive, particularly in difficult regimes (very bright/faint sources and high redshifts).
Feature Importance: SHAP analysis identifies WISE W1 and W2 magnitudes as the strongest individual predictors, consistent with previous literature. However, the model also leverages UV and optical bands for non-negligible refinements.
Computational Cost: The primary trade-off is inference time. With frozen weights, processing large support and target catalogs requires substantial GPU memory and time (e.g., ~76 minutes for $n_{train} \approx 121k$ ), suggesting that full-catalogue deployment may require support-set subsampling or distillation.

Significance and Claims
The paper concludes that TabPFN 2.5 is a strong default choice for probabilistic quasar photo-z estimation in S-PLUS-like surveys. Its significance lies in demonstrating that frozen, off-the-shelf foundation models can deliver calibrated, multimodal redshift densities without the need for survey-specific training or hyperparameter tuning. The authors emphasize that these models are particularly valuable when:

Training data is limited.
Calibration under covariate shift is critical.
Reducing catastrophic redshift failures in sparse regions of the data distribution is a priority.

The work provides the first systematic benchmark of tabular foundation models for this specific astronomical application, highlighting that while they do not universally dominate every metric (notably unweighted CDE loss against Flow-Spline), they offer the best overall balance of density quality, calibration, and robustness to selection biases. The authors caution that the results are specific to the S-PLUS quasar sample and that future work is needed to validate these findings on other object classes (galaxies, stars) and surveys.

Technical Summary: Tabular Foundation Models for Probabilistic Quasar Photometric Redshifts in S-PLUS

More like this