🔭 astrophysics

Uncertainties in Low-Count STIS Spectra

This paper evaluates the breakdown of standard "root-N" uncertainty calculations for low-count STIS spectra, introduces a new utility to compute accurate Poisson confidence intervals, and highlights a related software bug fix in the stistools package.

Original authors: Joshua D. Lothringer, Leonardo dos Santos, Joleen Carlberg, Sean Lockwood, Jacqueline Brown

Published 2026-01-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Joshua D. Lothringer, Leonardo dos Santos, Joleen Carlberg, Sean Lockwood, Jacqueline Brown

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Counting Invisible Fireflies

Imagine you are trying to count fireflies in a dark field at night.

The Bright Night: If there are thousands of fireflies, counting them is easy. You can say, "I counted 1,000," and you know your error margin is roughly the square root of that number (about 31). This is the standard rule astronomers have used for decades, called the "Root-N" rule. It works great when you have a lot of data.
The Dark Night: Now, imagine you are looking at a very dim star. You might only see zero or one firefly in a whole hour. In this situation, the old "Root-N" rule breaks down. It starts giving you impossible answers, like saying you have a "negative" number of fireflies or that your error is zero when you actually saw nothing.

This paper is about fixing the math used by the Hubble Space Telescope's STIS instrument when it looks at these very dim, "dark night" objects.

The Problem: The Old Ruler Doesn't Fit

The STIS instrument uses special detectors (called MAMA) that are incredibly sensitive. They are so good that they can detect single photons (particles of light) without the usual "static" or "noise" that other cameras have.

Because they are so quiet and sensitive, when astronomers look at very faint UV stars, the detectors often record 0 or 1 count.

The Issue: The standard software (the "pipeline") that processes this data still uses the old "Root-N" ruler.
The Result: When the count is near zero, the software says, "The error is tiny!" or even "The error is zero!" This is dangerous. It makes astronomers think they know the brightness of a star very precisely when, in reality, they might have seen nothing at all. It's like saying, "I saw zero apples, so I am 100% sure there are no apples," when in fact, you just might have missed them.

The Solution: A New, Better Ruler

The authors explain that when you have very few counts, you need to switch from the "Root-N" ruler to something called Poisson Confidence Intervals.

Think of it like this:

Root-N (Old Way): Assumes the data is perfectly symmetrical. If you count 10, the error goes up 3 and down 3.
Poisson (New Way): Acknowledges that you can't have negative counts. If you count 0, the error can't go down to -3. Instead, it says, "You saw 0, but there's a good chance the real number is actually 1 or 2." It gives you an upper limit (a ceiling) to tell you how bright the object could be, even if you saw nothing.

The paper notes that the COS instrument (a cousin to STIS on the same telescope) already fixed this problem in 2020. Now, the authors are bringing that same fix to STIS data.

The New Tools: What Astronomers Can Do Now

The authors didn't just point out the problem; they built new tools to help astronomers fix it themselves:

A New Calculator (stistools.poisson_err):
They created a simple computer tool (a Python function) that astronomers can run on their data. Instead of trusting the telescope's automatic error bars for dim stars, the astronomer can run this tool to get the "Poisson" error bars, which are much more honest about the uncertainty when counts are low.
A "How-To" Guide (Jupyter Notebook):
They made an interactive notebook (like a digital workbook) that walks users through the problem using real data from a star called GJ 436. It shows visually how the old method underestimates the error, and how the new method gives a more realistic range.

The Bug Fix: Don't Count the Same Firefly Twice

The paper also found and fixed a "bug" (a mistake in the code) in a tool called inttag.

The Mistake: This tool splits a long observation into smaller chunks. The old code tried to calculate the error for every single pixel in the image and then added them all up.
The Analogy: Imagine you are counting fireflies in a 5-pixel window. If you see 0 fireflies, the error for each pixel is high. If you add those errors together, you get a huge, scary number. But in reality, you are looking at the same group of fireflies across those pixels, not five different groups. Adding the errors made the uncertainty look way bigger than it actually was.
The Fix: The authors realized you shouldn't calculate the error pixel-by-pixel and then add them up. You should count the total fireflies in the whole window first, and then calculate the error. They fixed the code to stop doing the math wrong.

Summary

The Problem: When Hubble's STIS looks at very faint objects, it often sees 0 or 1 photon. The standard math used to calculate "how sure we are" (uncertainty) fails in these cases, making the data look more precise than it really is.
The Fix: The authors introduced a new statistical method (Poisson Confidence Intervals) that handles low numbers correctly.
The Tools: They released a new software tool and a guide so astronomers can manually apply this better math to their data.
The Bonus: They also fixed a bug where a different tool was accidentally making errors look too big by counting the same data multiple times.

In short: When the light is very dim, the old math lies. This paper gives astronomers a new, honest way to measure the darkness.

Technical Summary: Uncertainties in Low-Count STIS Spectra

Problem Statement
The Space Telescope Imaging Spectrograph (STIS) utilizes Multi-Anode Microchannel Array (MAMA) detectors for Far-Ultraviolet (FUV) and Near-Ultraviolet (NUV) observations. These detectors are characterized by extremely low dark current rates (approximately 0.0002 to 0.001 counts/sec/pix) and negligible read noise. While advantageous for sensitivity, these properties mean that observations of UV-dim sources can result in exposures with zero or one count in specific pixels, even during full-orbit exposures.

In such low-count regimes, the standard "root-N" approximation for calculating uncertainties (where error $\sigma \approx \sqrt{N}$ ) breaks down. This approximation assumes a Gaussian distribution, which is only valid when the number of counts $N \gg 10$ . For Poisson distributions at low $N$ , the distribution becomes highly asymmetric and discrete; the "left" side of the distribution cannot go below zero, causing the uncertainty to deviate significantly from the symmetric Gaussian prediction. Consequently, the current calstis pipeline, which relies on the root-N approximation, underestimates uncertainties in low-count regions, potentially leading to incorrect scientific inferences regarding the significance of non-detections or weak features.

Methodology
The authors evaluated the current behavior of the calstis pipeline by comparing its calculated uncertainties against manual calculations using Poisson confidence intervals. They utilized HST/STIS/G140M data from Program 12034 (observing the M-dwarf GJ 436), which contains both high-count Ly-α emission and regions of near-zero counts in the continuum.

The study involved:

Statistical Analysis: Comparing the root-N approximation against rigorous Poisson confidence intervals derived from the frequentist interpretation (specifically the Gehrels 1986 approximations and the astropy.stats.poisson_conf_interval module using the "frequentist-confidence" method).
Pipeline Verification: Confirming that the calstis pipeline currently applies the root-N formula ( $\sigma = \sqrt{(N-B)/g + (R/g)^2}$ ), which simplifies to $\sqrt{N}$ for MAMA data due to zero bias and read noise.
Tool Development: Creating a new utility, stistools.poisson_err, to manually calculate Poisson confidence intervals for 1D STIS spectra.
Bug Investigation: Analyzing a software defect in the stistools.inttag utility, which splits TIME-TAG data into sub-exposures. The investigation focused on how uncertainties are propagated when combining pixel-level errors versus summing counts before error calculation.

Key Contributions and Results

Validation of Pipeline Limitations: The analysis confirms that for wavelengths where the target flux drops to near-zero (e.g., longward of 1217 Å in the GJ 436 spectrum), the pipeline's root-N uncertainties are significantly underestimated compared to the true Poisson upper confidence limits. While the root-N error approaches zero as counts approach zero, the Poisson upper limit remains non-zero (e.g., $\lambda_{u, 1\sigma} \approx 1.866$ for $N=0$ ).
New Utility (stistools.poisson_err): The authors introduced a new function in the stistools Python package. This tool allows users to manually generate updated x1d or sx1 files containing four new columns: NET ERROR PCI UP, NET ERROR PCI DOWN, ERROR PCI UP, and ERROR PCI DOWN. These columns provide the 1 $\sigma$ Poisson upper and lower confidence intervals in both count-rate and flux units. The tool reconstructs the total observed counts (including background and dark) from the pipeline's error rates and exposure time, then applies the astropy "frequentist-confidence" interval.
Jupyter Notebook: A companion notebook titled "low count uncertainties" was released to the STScI HST Notebook Repository. It demonstrates the discrepancy between root-N and Poisson errors using the GJ 436 dataset and provides examples of how the CalCOS pipeline (which was updated in 2021) handles these uncertainties.
Resolution of stistools.inttag Bug: The authors identified and fixed a bug in the stistools.inttag utility. Previously, this tool calculated Poisson confidence intervals at the individual pixel level for sub-exposures and then combined them in quadrature. This approach was mathematically incorrect for Poisson statistics because the sum of quadrature-added Poisson intervals does not equal the Poisson interval of the sum. This led to a gross overestimation of errors in low-count sub-exposures. The fix (in stistools v1.4.5) reverts inttag to using the root-N approximation for sub-exposure error generation, deferring the application of Poisson intervals to the final extracted spectrum via the new poisson_err utility.

Significance
The paper claims that while the root-N approximation remains valid for high-count regions of STIS spectra, it is insufficient for the low-count regimes inherent to MAMA observations of faint UV sources. The significance of this work lies in providing observers with the necessary tools to accurately quantify uncertainties in these regimes, preventing the misinterpretation of noise as signal or the underestimation of error bars on weak detections.

By introducing stistools.poisson_err and correcting the inttag bug, the authors ensure that users can apply the same rigorous Poisson confidence interval methodology used in the CalCOS pipeline to STIS data. The authors emphasize that these tools are not automatically applied in the default calstis reduction but are available for manual application by users analyzing data in the low-count regime. The work serves as a case study for the proper statistical treatment of discrete, low-count photon data in astronomical instrumentation.