🔬 optics

Machine Learning in Application to Automatic Noise Processing of Solar Spectrograms

This paper presents a machine learning approach, specifically utilizing convolutional neural networks, to automate the cleaning, gap-filling, and transformation of solar spectrograms, thereby enabling the reliable analysis of previously unusable data such as edge regions and impurity-affected areas.

Original authors: I. I. Yakovkin, A. O. Bartenev, N. V. Petrova

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: I. I. Yakovkin, A. O. Bartenev, N. V. Petrova

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Sun is a star that never stops changing, and for astronomers, capturing its fleeting moments of violence is a race against time. When a solar flare erupts, it sends a burst of energy and light into space, offering a rare glimpse into the extreme physics of our star's atmosphere. To study these events, scientists have long relied on photographic plates, large sheets of light-sensitive material that record the Sun's light split into a rainbow of colors, known as a spectrum. These spectra act like a fingerprint, revealing the temperature, magnetic fields, and chemical makeup of the solar atmosphere. However, because these events happen only once and cannot be recreated, the original photographic plates are irreplaceable treasures. The challenge lies in turning these old, physical plates into digital data that modern computers can analyze. This process is not perfect; the way light passes through the plate to be scanned can create a clean image, while bouncing light off the surface can create a different image with its own set of flaws, such as blurring or ghostly halos. If scientists cannot trust the digital copy, they lose the ability to study the unique event it represents.

A team of researchers has now developed a new way to handle this problem using a type of computer program known as machine learning. Instead of trying to manually fix every scratch or dust spot on a digital image, they taught a computer to understand the relationship between two different ways of scanning the same solar plate. In their study, they focused on a specific solar flare that occurred on July 17, 1981, captured by a telescope in Kyiv. They took high-resolution scans of the same plate using two methods: one where light passed through the film (transmissive) and another where light reflected off the surface (reflective). The transmissive scans are generally clearer, but the reflective scans often contain extra reflections that create a blurred, shifted halo around the bright spots. The researchers wanted to see if a computer could learn to translate between these two versions, effectively cleaning up the messy reflective scan to look like the clean transmissive one, or vice versa.

To do this, they built a specialized digital brain called a convolutional neural network. This is a program designed to recognize patterns in images, much like how a human eye learns to recognize a face. The team fed the computer pairs of images: the messy reflective scan and the clean transmissive scan of the same solar flare. The computer was tasked with learning exactly how to transform one into the other. It learned that the "halo" seen in the reflective scan was essentially a smoothed-out, shifted version of the main image. By understanding this pattern, the computer could remove the blur from the reflective scan to reveal the clean spectrum underneath, or add the halo to the clean scan to see what the reflective version would look like. The results were striking. The computer could convert the images back and forth with such high accuracy that the difference between the computer's output and the real target image was less than one percent. This means the digital translation was nearly perfect, preserving the exact shape and brightness of the solar features.

Beyond simply translating images, this method offered a powerful new way to find and remove impurities. When the computer compared the two scans, it noticed that certain spots did not match the pattern it had learned. These mismatches appeared as bright spots in a map of errors, revealing the presence of dust, scratches, or other defects on the photographic plate. In the past, scientists had to look at every single image by hand to find these flaws, a slow and tedious process that could miss subtle errors. With this new approach, the computer automatically flagged the contaminated areas. The researchers tested this on a specific spectral line from the 1981 flare and found that the computer identified nearly all the dust particles and scratches that human experts had found, while also spotting twenty additional flaws that the humans had initially missed. These extra flaws were later confirmed to be real when the researchers looked closer, proving the computer's superior sensitivity.

The true value of this work lies in what it allows scientists to do with the data they already have. By using this machine learning tool, researchers can now confidently use parts of the spectrum that were previously considered too unreliable to analyze, such as the edges of the photographic plates where the image quality often degrades. This effectively increases the amount of usable data from these unique, one-time events. Furthermore, the method allows for the restoration of the original brightness of the spectrum in areas where dust or scratches had obscured the view. Instead of throwing away a damaged section of the image or guessing what it should look like, the computer can fill in the missing information based on the clean parts of the image. This ensures that the physical models scientists build to understand solar flares are based on the most complete and accurate data possible. The study demonstrates that while the original photographic plates are fragile and difficult to digitize perfectly, modern computing can bridge the gap, turning imperfect scans into reliable windows into the history of our Sun.

Technical Summary: Machine Learning in Application to Automatic Noise Processing of Solar Spectrograms

Problem Statement
Unique spectroscopic measurements of rare solar events, such as limb solar flares, are often captured on photographic plates (photoplates) which serve as irreplaceable historical data. These plates are digitized using various methods, primarily transmissive and reflective scanning. While transmissive scans are generally accurate, reflective scans often suffer from artifacts such as "halos" caused by additional reflections, blurring, and shifting. Conversely, both methods are susceptible to physical impurities like scratches and dust particles, which traditionally require manual inspection and exclusion. The challenge lies in the inability to easily transform between these different digitization formats to cross-validate data, and the lack of automated, reliable methods to detect and remove impurities without discarding valuable spectral information.

Methodology
The authors developed a Convolutional Neural Network (CNN) approach to address these challenges, specifically focusing on the limb solar flare observed on July 17th, 1981. The study utilized high-resolution grayscale scans of spectral lines from four photoplates, capturing moments at 8:17, 8:33, 9:02, and 9:51 UT. The training dataset consisted of scans of 8 spectral lines, while the test set specifically utilized the H $\gamma$ line at 8:33 UT and the H $\beta$ line at 9:51 UT.

Network Architecture: Two distinct CNN models were designed using the PyTorch framework with CUDA acceleration: one to transform transmissive scans to reflective scans ("transmissive-to-reflective") and another for the reverse ("reflective-to-transmissive").
- The architecture employs a dual-stream design. One stream reproduces the spectrum itself, while the other handles the addition or removal of the "halo" artifact.
- The "transmissive-to-reflective" model explicitly includes a shift block to account for the halo, allowing for smaller kernel sizes and improved performance.
- The "reflective-to-transmissive" model utilizes larger convolution kernels (up to 60x60) to handle the complexity of removing the halo without an explicit shift block.
- Pointwise operation blocks (chained sigmoids and linear transformations) are used to model the non-linear intensity relationships between scan types.
Training Strategy: Due to the limited number of image pairs (8 for training, 2 for testing), the authors employed a progressive validation approach: optimize model $\rightarrow$ train $\rightarrow$ test. This involved varying hyperparameters (kernel sizes, shifts, downsampling) on short training runs before a single extensive training run of 200,000 epochs.
Loss Function: The loss function is the Root Mean Square (RMS) of pixel-wise differences, calculated over the bottom 99.9% of pixels. This excludes the top 0.1% of losses (outliers), effectively training the model to focus on the uncontaminated majority of the image while ignoring impurities like scratches.
Optimization: The training alternated between the RProp and Adam optimization algorithms every 10,000 epochs.

Key Results

Transformation Accuracy: The CNNs successfully transformed spectra between reflective and transmissive formats with high fidelity. The "transmissive-to-reflective" model achieved a total loss of less than 0.5%, while the "reflective-to-transmissive" model achieved approximately 1%. The models preserved the intensity, position, width, and shape of spectral features (both emission and absorption peaks).
Efficiency: The study found that reversing a "transmissive-to-reflective" model to achieve a "reflective-to-transmissive" conversion was significantly faster computationally than training a direct model, while maintaining similar accuracy.
Impurity Detection: The pixel-wise loss maps generated by the models effectively highlighted impurities. Areas with scratches or dust particles exhibited high loss values, whereas the smooth background of the photoemulsion showed low loss.
Validation: A comparison between automated detection (using a threshold of 0.025 on normalized loss) and manual inspection of the H $\beta$ line at 9:51 UT showed that the automated method identified 30 out of 50 distinct high-loss locations. Of the 20 locations not initially detected manually, all were confirmed as valid impurities upon re-inspection. Conversely, 8 manually identified features were missed by the automated threshold but were visible in amplified loss maps, suggesting the threshold could be tuned for higher sensitivity.

Significance and Claims
The paper claims that this machine learning approach offers a practical technique for processing unique solar event spectrograms by:

Expanding Analyzable Data: Enabling the transformation between differently digitized spectra allows researchers to utilize data previously deemed unreliable (e.g., edges of spectrograms or reflective scans with halos), thereby increasing the number of spectral lines available for physical modeling.
Automating Noise Removal: The protocol automates the detection of impurities (scratches, dust) by leveraging the discrepancy between reflective and transmissive scans. This replaces the labor-intensive manual scrutiny required to ensure data integrity.
Restoring Data: By identifying contaminated pixels via the loss map, the authors propose a protocol to exclude these specific pixels during the averaging process used to derive spectral line profiles. This restores the profile shapes without the data loss or assumptions associated with interpolating entire contaminated segments.

The authors conclude that while the study is based on a specific dataset (the 1981 flare), the methodology is applicable to general spectroscopy for improving image quality, removing noise, and enhancing resolution.

More like this