PISP: Projected-Space Inference of Stellar Parameters

Technical Summary: Projected-Space Inference of Stellar Parameters (PISP)

Problem Statement
Large-scale spectroscopic surveys (e.g., SDSS, LAMOST, Gaia, DESI) generate vast datasets enabling the inference of high-dimensional stellar parameters, including effective temperature ($T_{\text{eff}}$), surface gravity ($\log g$), and abundances of dozens of elements. However, these parameters often exhibit significant physical and statistical correlations. Existing parameter-inference pipelines typically optimize directly in the original, high-dimensional parameter space without explicitly addressing these correlations. The authors observe that strong correlations can degrade optimization convergence speed and inference accuracy. While high-dimensional optimization strategies like linear and nonlinear embeddings exist in other fields, they have not been systematically applied to stellar parameter inference to mitigate these specific correlation-induced challenges.

Methodology
The paper proposes Projected-Space Inference of Stellar Parameters (PISP), a framework that maps standardized stellar parameters into an orthonormal projection space before optimization. The core methodology involves:

1. Basis Construction: An orthonormal basis is constructed using either Principal Component Analysis (PCA) or the Active Subspace (AS) method.
   • PCA: Derived from the covariance matrix of training parameters to statistically decorrelate variables.
   • AS: Derived from the gradients of the spectral emulator to identify directions in parameter space to which the spectrum is most sensitive.
2. Optimization Strategies: PISP optimizes the projection coefficients rather than the raw parameters. It implements four specific strategies:
   • Non-L1: Optimizes coefficients for a user-specified projected dimensionality ($n < 25$), effectively truncating low-variance or inactive directions.
   • L1: Introduces L1 regularization in the full projected space ($n=25$) to adaptively select relevant projection directions by inducing sparsity.
3. Implementation Variants:
   • PISP-CurveFit: Designed for CPU environments and fast single-spectrum inference, utilizing the scipy.optimize.curve_fit (trust-region reflective algorithm).
   • PISP-Adam: Designed for GPU-parallel large-scale inference, utilizing the Adam optimizer with PyTorch for batch processing.
4. Spectral Emulators: The framework is evaluated using two distinct neural network architectures trained on Kurucz synthetic spectra: a Fully Connected Neural Network (FNN) and a Residual Network (ResNet).

Key Contributions

• Framework Development: Introduction of PISP, which decouples the spectral emulator training from the inference optimization by operating in a transformed, projected space.
• Strategy Comparison: Systematic evaluation of four inference strategies (PCA-Non-L1, AS-Non-L1, PCA-L1, AS-L1) across different datasets and optimizers.
• Dual-Mode Implementation: Provision of both CPU-optimized (CurveFit) and GPU-optimized (Adam) versions to accommodate varying computational scales.
• Hyperparameter Tuning: Identification of dataset-specific optimal configurations, noting that synthetic data favors higher dimensions and lower regularization, while observed data benefits from dimensionality reduction and stronger regularization.

Results
The authors evaluated PISP on 14,623 Kurucz synthetic spectra (test set) and 722,896 APOGEE DR17 observed spectra, comparing performance against a baseline strategy (direct optimization in standardized parameter space).

• Synthetic Data (Kurucz):

  • PCA-L1 yielded the best performance. It reduced the standard deviation of differences ($\sigma(\Delta)$) by at least 0.01 dex for 12 of 20 elemental abundances across all emulator-optimizer combinations.
  • Significant improvements were observed for elements such as [N/H], [O/H], [Na/H], [Co/H], [P/H], [V/H], and [Cu/H], with reductions ranging from 0.05 to 0.72 dex.
  • PCA-L1 outperformed PCA-Non-L1, suggesting that low-variance basis vectors contain crucial information when training and test distributions are consistent.

• Observed Data (APOGEE DR17):

  • PCA-Non-L1 (with reduced dimensionality $n=7$) slightly outperformed PCA-L1, reducing $\sigma(\Delta)$ by $>30$ K for $T_{\text{eff}}$ and by at least 0.01 dex for 9 of 17 elemental abundances.
  • Notable reductions of 0.05–0.20 dex were seen for [O/H], [Na/H], and [V/H].
  • The authors attribute PCA-Non-L1's success on observed data to the robustness of focusing on high-variance directions, which may be less susceptible to noise and model-observation mismatches compared to the low-variance directions emphasized by L1 regularization.

• Efficiency:

  • PISP-Adam with PCA-Non-L1 achieved a $\sim4\times$ efficiency gain over the baseline for large-scale inference on APOGEE data.
  • PISP-Adam with PCA-L1 showed up to a $46\times$ speedup relative to PISP-CurveFit (baseline) when using the FNN emulator.
  • PISP reduces the number of iterations required for convergence in the Adam optimizer.

• Error Analysis:

  • PISP improved the consistency between model-derived errors and repeat-observation errors, particularly in the low signal-to-noise (S/N) regime where the baseline strategy tended to overestimate errors.

• Optimizer Dependence:

  • PISP's accuracy improvements were consistent across various optimizers (BFGS, L-BFGS, Adam, etc.).
  • For rotation-invariant optimizers (e.g., BFGS), the Non-L1 strategy (without dimensionality reduction) performed similarly to the baseline, confirming that the gains in Non-L1 for other optimizers stem from the decorrelation aiding non-invariant optimization paths.
  • PCA-L1 consistently outperformed other strategies across all optimizers due to the regularization constraints.

Significance and Claims
The paper claims that PISP provides a general, efficient, and accurate framework for high-dimensional stellar parameter inference. Its primary significance lies in:

1. Simultaneous Accuracy and Efficiency: PISP improves inference accuracy for multiple parameters while significantly reducing computational costs, making it suitable for the massive datasets of modern spectroscopic surveys.
2. Adaptability: The framework does not require retraining the spectral emulator; it acts as a post-training inference layer that can be integrated into existing pipelines.
3. Data-Dependent Strategy Selection: The work demonstrates that the optimal PISP configuration depends on the data distribution. PCA-L1 is preferred for matched synthetic distributions, while PCA-Non-L1 (with dimensionality reduction) is more robust for observed data with potential domain shifts.
4. Error Reliability: The method enhances the empirical consistency between estimated model errors and actual repeat-observation errors, providing more reliable uncertainty estimates for stellar parameters.

The authors conclude that PISP offers a practical solution for the next generation of high-dimensional stellar parameter inference in large spectroscopic surveys.