Spectral Data-cube Cleaning for CCAT Deep Spectroscopic Survey. I. Effect of correlated noise and filtering on the power spectrum

Imagine trying to hear a single, faint whisper in a crowded stadium during a thunderstorm. That is essentially the challenge facing astronomers who want to map the early universe. They are looking for a specific type of "light" (radio waves) emitted by ancient galaxies that formed billions of years ago, a time known as the Epoch of Reionization. To catch these whispers, they use giant telescopes equipped with super-sensitive detectors. However, the atmosphere above us is like a noisy, churning ocean of water vapor that constantly crackles and hisses, drowning out the cosmic signals. This "noise" isn't random; it's a low-frequency rumble that changes slowly over time, making it very hard to separate from the actual signal. To solve this, scientists use a technique called "Line-Intensity Mapping" (LIM). Instead of trying to take a sharp photo of individual distant galaxies (which are too faint to see clearly), they measure the total glow of all the galaxies in a patch of sky combined. It's like listening to the collective hum of a crowd rather than trying to hear one person's voice. The goal is to create a 3D map of the universe's structure, revealing how stars and gas were distributed when the cosmos was young.

This paper tackles the tricky business of cleaning up that noisy data. The authors simulated observations from a future instrument called EoR-Spec, which will be mounted on the Fred Young Submillimeter Telescope (FYST). They wanted to see if a specific set of cleaning tools, which they call a "Filter-and-Bin" (F&B) pipeline, could successfully strip away the atmospheric noise without accidentally throwing away the cosmic signal. Think of the data as a muddy stream; the pipeline is a series of sieves and filters designed to remove the mud (noise) while keeping the gold nuggets (the signal). The team ran thousands of computer simulations to test this pipeline. They found that the F&B pipeline is incredibly effective at scrubbing away the low-frequency atmospheric rumble, reducing it by about 10,000 times. However, like any strong filter, it has a side effect: it tends to smooth out the very largest, most spread-out patterns in the data. The results suggest that while this method is perfect for detecting the "grainy" details of the universe (where individual galaxies contribute to the noise), it struggles to recover the smooth, large-scale structures. But for the specific goal of measuring the "shot-noise" regime—the chaotic, small-scale clustering of gas and stars—the pipeline works beautifully, promising that the upcoming survey will be able to detect the combined glow of carbon and carbon monoxide from the early universe.

The Story of the Cosmic Whisper

The Setup: A Noisy Universe
Imagine you are trying to listen to a faint radio station, but your radio is sitting in a room where the air itself is buzzing with static. In astronomy, this static comes from the Earth's atmosphere, specifically water vapor. When the telescope looks up, it doesn't just see the stars; it sees the "breath" of the atmosphere, which fluctuates and creates a low-frequency hum known as $1/f$ noise. This noise is tricky because it mimics the slow, large-scale changes astronomers are trying to measure. If you don't clean it up, your map of the universe will just look like a blurry mess of atmospheric weather.

The Tool: The Filter-and-Bin Pipeline
To fix this, the authors designed a digital cleaning crew, a pipeline they call "Filter-and-Bin" (F&B). They tested this crew using a massive computer simulation that acted like a virtual telescope. They fed the simulation with a fake universe containing the signals they wanted to find (light from carbon and carbon monoxide gas) and the annoying atmospheric noise. Then, they ran the data through four distinct cleaning steps:

1. Polynomial De-trending: First, they removed the slow, lazy drifts in the data, like smoothing out a wobbly table leg.
2. Scan-Synchronous Correction: Next, they fixed the patterns caused by the telescope moving back and forth. It's like realizing the static gets louder every time you turn your head, and subtracting that specific pattern.
3. Common-Mode Subtraction: Since the atmospheric noise hits all the detectors at once, they calculated the "average noise" and subtracted it from everyone's data. This is like a choir director asking everyone to stop humming the same background note.
4. PCA Filtering: Finally, they used a smart math trick called Principal Component Analysis (PCA) to find and remove the remaining stubborn patterns that the other steps missed. They removed the top three "patterns" of noise, leaving mostly random, harmless static behind.

The Results: A Clean Signal with a Catch
The simulation showed that this cleaning crew was a huge success. At the lowest frequencies (the slowest, most annoying rumbles), the pipeline reduced the noise by about four orders of magnitude (that's 10,000 times quieter!). The remaining data looked almost perfectly like random white noise, which is exactly what you want for a clean measurement.

However, there is a trade-off. Just as a sieve lets sand through but stops pebbles, this pipeline lets the small, detailed signals through but blocks the very largest, smoothest waves. The authors found that for small-scale details (where the wave number $k$ is greater than 0.5 Mpc$^{-1}$), the pipeline keeps more than 80% of the signal intact. But for the largest, most spread-out structures (where $k$ is less than 0.1 Mpc$^{-1}$), the pipeline suppresses the signal so much that less than 20% remains.

What This Means for the Future
The paper concludes that this F&B pipeline is the right tool for the job the CCAT Deep Spectroscopic Survey (DSS) is planning. The survey aims to map the universe at redshifts between 3.5 and 8.0 (a time when the universe was much younger). In their simulations, they found that with a single telescope module operating at 50% efficiency, they can detect the signal on small scales. If they use two modules at 100% efficiency, they can detect the signal across all scales they are looking at.

The authors are careful to note that while this pipeline is great for the "shot-noise" regime (the chaotic, small-scale clumping of gas), it isn't the best for measuring the smooth, large-scale structure of the universe. For that, they will need even more advanced methods in the future. But for now, this pipeline gives them a solid, reliable way to start listening to the whispers of the early universe, filtering out the storm so the cosmic story can finally be heard.