Euclid. Properties and performance of the NISP signal estimator

Technical Summary: Properties and Performance of the NISP Signal Estimator

Problem Statement
The Euclid spacecraft's Near-Infrared Spectrometer and Photometer (NISP) instrument faces stringent constraints regarding onboard computational resources and telemetry bandwidth. To manage the massive data volume generated by its 16-detector mosaic (approx. 67 million pixels), NISP employs an onboard signal estimator algorithm to process near-infrared data in real-time. This algorithm reduces data volume by estimating flux and a quality factor (QF) directly on the Data Processing Units (DPUs), rather than downlinking raw frames.

However, to meet hardware limitations, the estimator relies on specific simplifications: it operates in the signal difference space, assumes a diagonal covariance matrix (neglecting off-diagonal correlations), treats the photon-noise component as a free parameter, and utilizes detector-averaged values for readout noise and conversion gain instead of pixel-specific maps. These approximations introduce potential systematic biases in flux estimation and affect the statistical reliability of the QF parameter, which is used to flag anomalies such as cosmic ray hits. While the estimator was theoretically derived in previous work (Kubik et al. 2016b), its performance under actual flight conditions and the precise quantification of biases introduced by onboard approximations required further investigation.

Methodology
This study evaluates the NISP signal estimator through a combination of theoretical re-analysis, Monte Carlo (MC) simulations, and empirical analysis of early flight data.

1. Theoretical and Simulation Analysis: The authors revisited the statistical foundations of the likelihood function used in the estimator. They investigated the validity of the Gaussian approximation for group differences, the impact of neglecting off-diagonal covariance terms, and the "folding bias" that occurs at low flux levels when negative group differences are squared. MC simulations were conducted across various flux regimes (dark current, zodiacal background, and flat field) and readout noise levels to model the behavior of the signal estimator and QF under ideal conditions versus conditions mimicking the onboard Data Processing Approximation (DPA).
2. Flight Data Analysis: The performance was validated using NISP Level 1 (LE1) data acquired during the Performance Verification and early survey phases (September 2023 – April 2024). This dataset includes dark exposures and scientific observations (spectroscopic grisms and photometric filters).
3. Reference Signal Construction: To quantify bias without knowing the true incident flux a priori, the authors utilized "science-pixels rawlines"—subsets of full-frame single-readout data downlinked for a small fraction of pixels. They applied a linear least-squares fit (LSF) to these rawlines to generate an unbiased reference signal ($g_{LSF}$). The bias of the onboard estimator was then calculated as the deviation of the estimator's output from this LSF reference.
4. QF Analysis: The distribution of the QF parameter was analyzed to assess its sensitivity to the DPA and its ability to detect non-linearities and cosmic ray hits, comparing flight data distributions against theoretical $\chi^2$ expectations.

Key Contributions

• Analytical Derivation of Bias and Variance: The paper derives a closed-form analytical expression for the constant bias of the signal estimator at high flux levels (Eq. 18) and provides a new, improved analytical expression for the variance of the estimator (Eq. 19). The latter corrects previous approximations that neglected off-diagonal covariance elements, offering a more accurate uncertainty estimate, particularly when the covariance matrix is not pre-computed.
• Quantification of Data Processing Approximation (DPA) Effects: The study isolates and quantifies the systematic bias introduced by using detector-averaged readout noise and conversion gain values instead of pixel-specific maps. It demonstrates that while these approximations introduce bias, the magnitude is predictable and controllable.
• Statistical Characterization of QF: The authors derive the expected value and variance of the QF parameter, confirming it follows a scaled $\chi^2$ distribution. They establish the relationship between the QF bias and the signal estimator bias, showing that the QF expectation value is $n_g - 2$ under ideal conditions.
• Flight Performance Validation: The paper provides the first comprehensive validation of the NISP signal estimator using in-flight data, confirming that the theoretical models and simulations accurately predict the instrument's behavior.

Results

• Signal Bias: For 99% of the NISP pixel array, the systematic bias of the signal estimator is lower than $0.01 \, e^- s^{-1}$ across the relevant flux range ($f \gtrsim 0.01 \, e^- s^{-1}$). This bias is well within the noise budget of the estimated signal.
• DPA Impact: The bias induced by using detector-averaged readout noise ($\sigma_{R, DPU}$) increases with the deviation of a pixel's actual noise from the average. However, for the vast majority of pixels, this effect remains negligible. The analysis confirms that the bias is flux-dependent in specific regimes but largely constant at high flux.
• Variance Accuracy: The newly derived variance estimator (Eq. 19) shows perfect agreement with observed dispersion for fluxes above $0.5 \, e^- s^{-1}$, independent of readout noise and readout mode.
• QF Performance: The QF parameter effectively identifies anomalies. In spectroscopic mode, the QF distribution follows the expected $\chi^2$ distribution at high flux. At lower fluxes, deviations occur due to the DPA and the dominance of readout noise. The study confirms that the QF is highly sensitive to cosmic ray hits, which manifest as outliers in the distribution. The fraction of pixels flagged as outliers due to DPA alone is less than 1% in spectroscopic mode and up to ~3.5% in photometric mode under dark conditions, but these are distinguishable from space weather-induced outliers.
• Space Weather: The flight data reveals that the tail of the QF distribution is significantly influenced by the space radiation environment (solar energetic particles), which can flag a large percentage of pixels during solar events, a factor not present in ground-based simulations.

Significance
The paper concludes that the NISP signal estimator performs robustly in flight, meeting the Euclid mission's stringent data-quality requirements. The findings corroborate previous theoretical results and demonstrate that the simplifications required for onboard processing do not introduce uncontrolled systematics. The derived analytical corrections for bias and variance provide a robust framework for the Science Ground Segment (SGS) to interpret the data accurately. Furthermore, the study validates the QF parameter as a reliable diagnostic tool for data quality, capable of distinguishing between instrumental approximations and genuine physical anomalies like cosmic ray hits. The agreement between flight data and simulations suggests that the theoretical modeling of the signal estimator is sound, allowing for the derivation of leading-order corrections to systematic biases where necessary.