💻 computer science

MATANet: A Multi-context Attention and Taxonomy-Aware Network for Fine-Grained Underwater Recognition of Marine Species

This paper proposes MATANet, a novel framework that integrates multi-context environmental attention and taxonomy-aware hierarchical supervision to significantly improve fine-grained recognition of marine species in challenging underwater scenes, achieving state-of-the-art performance and first place in the FathomNet 2025 Challenge.

Original authors: Donghwan Lee, Byeongjin Kim, Geunhee Kim, Hyukjin Kwon, Nahyeon Maeng, Wooju Kim

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Donghwan Lee, Byeongjin Kim, Geunhee Kim, Hyukjin Kwon, Nahyeon Maeng, Wooju Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to identify a specific type of sea anemone in a crowded underwater photo. If you only look at the anemone itself (like zooming in tight on its face), it might look very similar to dozens of other anemones. But if you step back and look at the whole picture, you might notice it's sitting on a specific type of rock, surrounded by a specific kind of coral, or living next to a particular fish. An expert marine biologist uses all these clues—the creature itself plus its neighborhood and its family tree—to make the right call.

This paper introduces MATANet, a computer program designed to think exactly like that expert. It's a new way for AI to recognize marine life with high precision, even when the creatures look almost identical.

Here is how it works, broken down into simple concepts:

1. The Problem: "Zooming In" Isn't Enough

Most current AI models for underwater photos act like a detective who only looks at the suspect's face. They focus strictly on the "Region of Interest" (the animal itself).

The Flaw: In the ocean, many animals look very similar. Without seeing their surroundings (the "context") or knowing their family history (the "taxonomy"), the AI gets confused. It's like trying to tell two identical twins apart without knowing their names or seeing where they live.

2. The Solution: MATANet's Two Superpowers

The researchers built MATANet with two special tools to fix this:

Tool A: The "Neighborhood Watch" (Multi-Context Environmental Attention)

Instead of just looking at the animal, this module looks at the animal and its surroundings at different distances.

The Analogy: Imagine you are identifying a person in a crowd.
- Level 1: You look at their face (the animal).
- Level 2: You look at who is standing right next to them (the immediate habitat).
- Level 3: You look at the whole room they are in (the broader environment).
How it helps: The AI learns that a certain type of crab is always found near specific rocks, while another looks the same but lives near coral. By "paying attention" to these surroundings, the AI can spot the differences that a zoomed-in view misses.

Tool B: The "Family Tree" (Taxonomy-Aware Learning)

Biological classification is organized like a family tree: Kingdom → Phylum → Class → Order → Family → Genus → Species.

The Analogy: Think of a library. If you are looking for a specific book, it helps to know it belongs to the "Science Fiction" section, then the "Space Opera" subsection, before you even look at the title.
How it helps: MATANet doesn't just try to guess the specific species name immediately. It uses the family tree as a "training guide." It forces the AI to learn that a "Lion" and a "Tiger" are more similar to each other than they are to a "Shark." This helps the AI organize its knowledge so it doesn't mix up closely related species.

3. The Results: Winning the Underwater Olympics

The team tested MATANet on three different datasets:

FathomNet 2025: A massive collection of high-quality underwater photos.
FishCLEF2015: A dataset of fish in coral reefs.
FAIR1M: A dataset of satellite images (to see if the method works outside the ocean).

The Outcome:

MATANet beat all other existing methods on the underwater datasets.
It took 1st Place in the FathomNet 2025 Challenge (a major competition held at the CVPR 2025 workshop).
Even on the satellite images (where the "neighborhood" is different), it performed very well, proving the idea is flexible.

4. What the AI Actually "Sees"

The researchers looked at the AI's "attention maps" (heatmaps showing what the computer is looking at).

Success: When identifying a specific octopus, the AI didn't just look at the octopus; it also highlighted the surrounding water and background, ignoring irrelevant objects like a robotic arm in the photo.
Failure: Sometimes, if the image is blurry or the background is too confusing, the AI gets distracted by irrelevant things, just like a human might.

Summary

MATANet is a smarter way to teach computers to recognize sea creatures. Instead of just memorizing what a fish looks like in isolation, it teaches the AI to:

Look at the neighborhood (context).
Understand the family tree (taxonomy).

By combining these two things, the AI becomes much better at telling apart the "twins" of the underwater world, which is crucial for scientists trying to count species and protect the ocean.

Technical Summary: MATANet for Fine-Grained Underwater Recognition

Problem Statement
Fine-grained visual recognition (FGVC) of marine organisms is critical for ecological research, biodiversity monitoring, and conservation policy. However, existing approaches often rely heavily on object- or Region of Interest (ROI)-centered representations, which may fail to capture the interactions between target organisms and their surrounding environments. Furthermore, the hierarchical structure of biological taxonomy is frequently underutilized in representation learning. These limitations hinder discriminative performance in challenging underwater scenes where visually similar organisms appear under diverse environmental conditions. Additionally, the scarcity of taxonomists capable of reliable manual identification necessitates robust, automated systems for scalable analysis.

Methodology
The authors propose MATANet (Multi-context Attention and Taxonomy-Aware Network), a framework designed to mimic expert taxonomic identification practices by jointly modeling organism-level morphology, environmental context, and taxonomic hierarchy. The architecture comprises two primary modules:

Multi-Context Environmental Attention Module (MCEAM):
- Input: The module processes the primary ROI (target organism) alongside multiple ROI-centered contextual regions at different scales (e.g., 3×, 5×, and full-image scales).
- Mechanism: Using a shared Vision Transformer (ViT) backbone (initialized with DINOv2 weights), MCEAM employs a cross-attention mechanism. The ROI embedding serves as the query, while patch embeddings from the multi-scale context regions serve as keys and values.
- Function: This allows the model to selectively attend to informative environmental cues (such as substrate, nearby organisms, and habitat configurations) conditioned on the target organism's representation. The resulting context-aware features are concatenated with the ROI embedding and projected into a unified latent space.
Hierarchy-Aware Representation Learning Module (HRLM):
- Mechanism: HRLM utilizes taxonomic hierarchy (e.g., Order, Family, Genus) as auxiliary supervision during training. It does not function as a hierarchical classifier for final inference but acts as a regularizer for the embedding space.
- Function: A transformed embedding is fed into level-specific auxiliary classifiers. The module computes a hierarchical loss ( $L_{hier}$ ) based on valid higher taxonomic levels, encouraging the learned embeddings to reflect taxonomic relationships across levels without enforcing hard prediction constraints at the final output.
- Objective: The total training objective combines the main classification loss ( $L_{cls}$ ) on the finest available target label with the auxiliary hierarchical loss ( $L_{total} = L_{cls} + \lambda L_{hier}$ ).

Key Contributions

MCEAM: Introduces a mechanism to model cross-attention between the primary object region and multi-scale environmental context, addressing the limitations of ROI-centric pipelines by integrating local morphological details with habitat-level cues.
HRLM: Designs a representation learning strategy that uses taxonomic hierarchy as level-wise auxiliary supervision to regularize the embedding space, promoting semantically structured representations that reflect biological relationships.
Comprehensive Evaluation: Extensive experiments are conducted on FathomNet2025 and LifeCLEF2015-Fish (marine datasets) and FAIR1M (remote sensing dataset) to evaluate performance across diverse resolutions, environmental complexities, and label structures.
Benchmark Achievement: The proposed method achieved the first-place ranking in the FathomNet 2025 Challenge at the CVPR 2025 FGVC12 workshop.

Experimental Results

FathomNet2025: MATANet achieved the lowest Hierarchical Distance (HD) on both Public and Private test sets, with a weighted average HD of 1.658 (M4 configuration) and 1.54 (M5 configuration with a larger backbone), outperforming strong baselines including ResNet, Swin Transformer, and specialized FGVC models like MSDBN and TransFG.
LifeCLEF2015-Fish: The model demonstrated consistent improvements, achieving an Accuracy (ACC) of 0.787 and an HD of 1.144, outperforming all benchmark models. The performance gap was noted to be smaller than on FathomNet2025, likely due to the lower resolution and simpler taxonomic structure of this dataset.
FAIR1M (Cross-Domain): In a non-underwater setting, MATANet achieved the highest ACC (0.7395) and lowest HD (0.973) among compared models, suggesting the framework's applicability beyond marine imagery, though the benefit of contextual cues was less pronounced due to limited background diversity in the dataset.
Ablation Studies:
- Contextual information (MCEAM) significantly reduced HD compared to ROI-only baselines.
- HRLM further improved performance by structuring the embedding space, outperforming other hierarchy-aware strategies like HXE and SoftLabel when used as auxiliary supervision rather than a replacement for the main loss.
- Visualization via t-SNE confirmed that HRLM encourages the formation of compact, taxonomy-aware clusters in the embedding space.

Significance and Claims
The paper claims that MATANet effectively addresses the challenges of fine-grained marine recognition by moving beyond isolated object features to incorporate the ecological context and biological taxonomy that experts utilize. The authors assert that:

Context is Critical: Environmental context provides complementary cues that improve discrimination, particularly for context-sensitive species, though it can introduce ambiguity for some categories.
Structure Matters: Using taxonomic hierarchy as auxiliary supervision, rather than just a prediction target, helps learn more structured and discriminative feature representations.
Practical Impact: The framework's success in the FathomNet 2025 Challenge demonstrates its viability for large-scale, automated biodiversity assessment.

The authors modestly acknowledge limitations, noting that the method currently relies on pre-existing ROI annotations (requiring a detection step in "in-the-wild" scenarios) and that fixed context scales may not be optimal for all datasets. They also note that contextual cues are not universally beneficial and can sometimes introduce noise or ambiguity.