💻 computer science

Landsat-Sentinel-2 Algal Bloom Mapping Using Vision Transformers: Model Description, Implementation, and Examples

This study demonstrates that vision transformer-based deep learning models, particularly the Swin Transformer, effectively map fragmented coastal algal blooms using harmonized 30-m Landsat and Sentinel-2 imagery, outperforming traditional spectral indices and coarse-resolution sensors in accuracy and robustness against atmospheric interference.

Original authors: Thainara Lima, Vitor Martins

Published 2026-06-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Thainara Lima, Vitor Martins

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Finding "Green Soup" from Space

Imagine the ocean is a giant, clear swimming pool. Sometimes, tiny plants called algae grow so fast they turn the water into "green soup" (algal blooms). These blooms can be harmful to fish, wildlife, and people.

Scientists have been trying to spot this "green soup" from space for a long time. They use satellites, which are like giant cameras in the sky. However, the old cameras (satellites) are a bit blurry. They can see big, massive clouds of algae, but they miss the small, scattered patches that look like little islands of green in the water.

This paper is about a new, sharper way to find these small patches using a special kind of "smart camera" (Artificial Intelligence) and two specific satellites: Landsat and Sentinel-2.

The Problem: The "Blurry" Old Way

For years, scientists used a simple math trick to find algae. It's like trying to find a specific shade of green in a painting by just looking at the color numbers.

The Issue: Sometimes the sun reflects off the water (glint) or clouds pass by, making the water look green even when there is no algae. The old math tricks get confused by these tricks of light and often say, "That's algae!" when it's just a shiny reflection.
The Resolution: The old satellites are like looking at a map from 10,000 feet up. You can see the continent, but you can't see the individual houses. The new satellites (Landsat and Sentinel) are like flying lower, at 1,000 feet, where you can see the houses. But the "math trick" still struggles to tell the difference between a house and a tree.

The Solution: Teaching a Robot to "See" Like a Human

Instead of using a simple math formula, the researchers taught a computer to learn what algae looks like, just like a human learns to recognize a dog by seeing many pictures of dogs.

They used a type of AI called a Vision Transformer.

The Analogy: Imagine you are trying to find a specific type of cloud in the sky.
- The Old Way (Math Formula): You measure the exact shade of white. If it's white enough, you call it a cloud. But fog is also white, so you get it wrong.
- The New Way (Vision Transformer): You look at the shape and the texture. You see that clouds are fluffy and have edges, while fog is flat and blurry. The AI looks at the whole picture at once, understanding how the "green soup" connects and flows, rather than just checking one pixel at a time.

What They Did (The Recipe)

Gathering the Ingredients: They collected thousands of photos from the Landsat and Sentinel satellites taken all over the world in 2023 and 2024. They focused on places known to have algae.
Cleaning the Photos: They used a special tool (called AQUAVis) to clean the photos. This tool removes the "noise" caused by the atmosphere, clouds, and sun glare, making the water look as clear as possible.
Training the AI: They showed the AI millions of tiny square pieces of these photos (called "patches"). They told the AI: "This green patch is algae," and "This blue patch is just water."
The Contest: They tested five different types of AI "brains" to see which one was the best at finding the algae.
- One was a classic brain (ResUNet).
- Four were new, fancy brains based on the "Transformer" technology (like the ones used in chatbots, but for pictures).

The Results: Who Won?

The Winner: The Swin Transformer was the best at the job. It was like the most careful detective. It could find the small, scattered patches of algae that the others missed, and it didn't get tricked by the sun's glare.
The Runner-Up: The classic brain (ResUNet) was also very good and very fast.
The Losers: Some of the other fancy AI models got confused. One of them (Prithvi) was like a student who studied a textbook on land animals but tried to identify fish; it didn't know enough about water to do the job well.

Why This Matters (The "Aha!" Moment)

The researchers compared their new AI method against the old "math formula" method and a lower-resolution satellite (MODIS).

The Glint Test: When the sun was shining brightly on the water (creating a glare), the old math formulas screamed, "ALGAE!" everywhere the sun hit. The new AI, however, looked at the shape and said, "No, that's just a shiny reflection." It avoided the false alarms.
The Detail Test: The old satellite (MODIS) saw the ocean as a few big, blurry blobs. The new method (30-meter resolution) saw the ocean as a detailed mosaic. It found tiny, broken-up islands of algae that the old satellite completely missed.

The Bottom Line

This paper proves that we can now use advanced AI to look at medium-sized satellites (Landsat and Sentinel) and find small, tricky patches of algae in coastal waters with high accuracy. It's a step forward from "guessing based on color" to "understanding based on shape and context."

What the paper does not claim:

It does not claim this system is running 24/7 for emergency alerts right now (though it paves the way for it).
It does not claim to identify the specific type of algae (like "this is a toxic red tide" vs. "this is harmless green algae"). It just finds where the algae is.
It does not claim to replace all other methods, but rather shows that this new AI approach is better than the old math tricks for finding fragmented blooms.

Technical Summary: Landsat-Sentinel-2 Algal Bloom Mapping Using Vision Transformers

Problem Statement
Coastal algal bloom monitoring requires frequent, spatially detailed, and globally consistent observations. While coarse-resolution ocean-color sensors (e.g., MODIS, OLCI) provide broad synoptic coverage, their moderate spatial resolution (30–250 m) limits the detection of small, fragmented, or low-concentration blooms that often form as clumps or rafts smaller than a single pixel. Furthermore, retrieving accurate water reflectance in coastal zones is challenging due to bottom reflectance, high suspended sediment loads, and land adjacency effects. Traditional bio-optical methods and spectral indices (e.g., NDVI, FAI, MCI) rely on empirical thresholds and often fail to distinguish algal blooms from atmospheric artifacts like clouds and sun glint, or from non-algal features with similar spectral signatures. Although deep learning (DL) has shown promise in remote sensing, its application to algal bloom mapping using medium-resolution (30 m) Landsat-8/9 and Sentinel-2 data remains emergent, with a lack of harmonized reflectance products, standardized labeling protocols, and publicly available reference datasets.

Methodology
This study presents a comprehensive framework for 30-m coastal algal bloom mapping using harmonized Landsat-8/9 OLI and Sentinel-2 MSI data.

Data Harmonization and Pre-processing: The authors utilized the AQUAVis pipeline to generate a harmonized remote sensing reflectance ( $R_{rs}$ ) product. This pipeline applies atmospheric correction (6SV), glint and adjacency effect mitigation, and sensor-specific spectral bandpass adjustments. The output consists of four spectral bands (Blue, Green, Red, NIR) at 30-m resolution, along with a quality assurance (QA) band for cloud/shadow masking.
Dataset Generation: A globally distributed dataset was created from 225 identified bloom hotspots.
- Training: 660 images from 2023 were processed into 24,265 training patches (256×256 pixels).
- Validation: 186 images from 2024 were processed into 12,255 validation patches to assess temporal transferability.
- Labeling: A semi-automatic workflow was employed. Initial masks were generated using a dynamic NDVI threshold and unsupervised object detection. These were manually refined using false-color composites (Red, NIR, Green) to correct for spectral index instability and ensure labels reflected morphological bloom structures (e.g., filamentous patterns) rather than just spectral magnitude. Input features included the four $R_{rs}$ bands plus NDVI and Floating Algae Index (FAI) as additional channels.
Model Architectures: Five deep learning models were implemented and compared:
1. ResUNet: A CNN-based encoder-decoder with context encoding and residual connections (baseline).
2. Vanilla Vision Transformer (ViT): A standard transformer architecture with self-attention mechanisms.
3. Swin Transformer: A hierarchical ViT using window-based multi-head self-attention to reduce computational complexity and capture local context.
4. SegFormer: A hybrid model combining a Mix Transformer (MiT) encoder with a lightweight decoder.
5. Prithvi (MAE): A foundational model (IBM/NASA) pre-trained on Harmonized Landsat-Sentinel data, fine-tuned with a SegFormer decoder.
Training Strategy: Models were trained using AdamW optimizers, composite loss functions (Dice, Cross-Entropy, IoU), and distance-based weighting maps to emphasize boundaries. Pre-trained weights (ImageNet-1k or Prithvi) were used to facilitate convergence.

Key Contributions

First Application of ViTs: This is the first study to apply Vision Transformer architectures (Vanilla ViT, Swin, SegFormer) to 30-m coastal algal bloom segmentation.
Global Reference Dataset: The development of a globally distributed, 30-m algal bloom mask dataset (36,520 patches) covering diverse optical water types and bloom morphologies, accompanied by a scalable semi-automatic labeling workflow.
Operational Framework: The establishment of a pipeline for consistent, medium-resolution bloom monitoring using the Harmonized Landsat-Sentinel (HLS) virtual constellation, moving beyond region-specific approaches.

Results

Model Performance: All deep learning models demonstrated strong capabilities in detecting floating algal blooms.
- ResUNet achieved the highest overall F1-score (0.58) with balanced omission (9.88%) and commission (56.59%) errors.
- Swin Transformer performed comparably to ResUNet (F1-score 0.57) and showed superior robustness in handling spatial fragmentation.
- SegFormer achieved the highest recall (0.92) but suffered from high commission errors (64.96%), indicating a tendency to over-segment.
- Vanilla ViT and Prithvi (MAE) showed lower performance, with Prithvi exhibiting the highest omission error (33.26%), likely due to the mismatch between its pre-training domain (general Earth observation) and the specific spectral/spatial characteristics of aquatic blooms.
Environmental Robustness: The Swin Transformer and ResUNet models maintained performance across diverse Optical Water Types (OWTs), including turbid and clear waters. Crucially, under challenging atmospheric conditions (thin clouds and intense sun glint), the deep learning models effectively avoided false positives that plagued traditional spectral indices (NDVI and FAI), which produced widespread false detections in glint-affected areas.
Resolution Advantage: Comparisons with 1-km MODIS-derived products revealed that the 30-m resolution models could detect fragmented and irregular bloom structures invisible to coarser sensors. While MODIS provided a lower-bound estimate of bloom extent, the 30-m maps revealed significantly more detailed spatial patterns.

Significance and Claims
The authors claim that this study establishes a new frontier for deep learning in aquatic remote sensing. By demonstrating that Vision Transformers can successfully delineate fine-scale bloom features in dynamic coastal environments, the work supports the transition from coarse, global monitoring to scalable, medium-resolution operational systems. The study highlights that deep learning models, by leveraging spectral-spatial context, can overcome the limitations of traditional threshold-based indices, particularly in distinguishing true blooms from atmospheric and surface noise. The authors position their framework as a foundational step toward a potential operational 30-m algal bloom mapping system, offering improved early-warning capabilities and ecological assessments for coastal management. They acknowledge that while the models perform well, future work is needed to develop domain-specific foundational models trained on aquatic datasets to further improve generalization and reduce reliance on pre-trained models from non-aquatic domains.