💻 computer science

Mine-JEPA: In-Domain Self-Supervised Learning for Mine-Like Object Classification in Side-Scan Sonar

The paper introduces Mine-JEPA, an in-domain self-supervised learning pipeline that outperforms large general-purpose foundation models like DINOv3 in side-scan sonar mine classification by achieving superior accuracy with significantly fewer parameters and training on only 1,170 unlabeled images.

Original authors: Taeyoun Kwon, Youngwon Choi, Hyeonyu Kim, Myeongkyun Cho, Junhyeok Choi, Moon Hwan Kim

Published 2026-04-02

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Taeyoun Kwon, Youngwon Choi, Hyeonyu Kim, Myeongkyun Cho, Junhyeok Choi, Moon Hwan Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to find hidden treasure (underwater mines) on the ocean floor using a special camera called Side-Scan Sonar.

The problem? The ocean floor looks nothing like the world we see with our eyes. To a human, a mine looks like a shadowy, grainy blob on a black-and-white photo. To a standard computer vision AI (trained on millions of photos of cats, cars, and sunsets), it looks like complete gibberish.

Furthermore, we have a massive shortage of "labeled" photos. We don't have thousands of experts to draw boxes around every mine in every picture. We only have a tiny scrapbook of about 1,170 images.

This is where Mine-JEPA comes in. It's a new, clever way to teach computers to be mine detectives without needing a massive library of textbooks.

Here is the story of how it works, explained with simple analogies:

1. The Problem: The "Foreign Language" Barrier

Think of standard AI models (like the famous DINOv3) as brilliant students who have read every book in the world's largest library (1.7 billion images). They are experts at recognizing natural things.

But if you hand them a sonar image, it's like handing a Shakespeare expert a book written in a completely alien language. Even though the expert is smart, they don't understand the "grammar" of sonar (acoustic shadows and textures). If you try to force them to learn this new language using only a few pages, they might get confused and actually perform worse than before.

2. The Solution: The "Apprentice" Approach

Instead of hiring the "Super Expert" (DINOv3) and trying to retrain them, the authors built a new system called Mine-JEPA.

Think of Mine-JEPA as a smart apprentice who starts with a basic understanding of shapes and edges (from a standard image library called ImageNet) but is specifically trained to learn the "dialect" of the ocean floor.

They used a special teaching method called SIGReg.

The Analogy: Imagine you are trying to learn a new song. Instead of listening to the song over and over (which is boring and hard with few examples), you are given a "sketch" of the melody and asked to fill in the blanks.
How it works: The AI looks at a sonar patch, creates a few "distorted" versions of it (like flipping it or changing the brightness), and tries to make sure its brain recognizes them all as the same object. At the same time, it ensures it doesn't just memorize the noise. It learns the essence of the shape.

3. The Secret Sauce: "Sonar-Specific" Training

The authors realized that you can't teach a sonar AI using the same tricks used for regular photos.

The Mistake: If you take a photo of a cat and randomly change its colors (hue/saturation), it's still a cat. But sonar images are grayscale acoustic maps. Changing the "colors" (which are actually signal strengths) destroys the information.
The Fix: They created a "Sonar Gym" for the AI. They only allowed changes that make sense for the ocean: flipping the image upside down (since the sonar could be looking from above or below) or rotating it slightly. They stopped messing with colors because, in sonar, "color" doesn't exist.

4. The Surprise Discovery: "Less is More"

Here is the most surprising part of the story.

The researchers tried taking the "Super Expert" (DINOv3) and trying to teach it the sonar language using their new method. It failed. The expert got confused and its performance dropped by 10–13%. It turns out, when you are already an expert in one field, trying to force a completely new, weird field on you can break your brain.

However, the Apprentice (Mine-JEPA), starting with a simpler foundation and learning specifically for the ocean, beat the Super Expert.

The Result: Mine-JEPA, trained on just 1,170 images, achieved a higher accuracy than DINOv3, which was trained on 1.7 billion images.
The Efficiency: They even tested a tiny version of their model (ViT-Tiny). It was 4 times smaller than the giant foundation models but still did the job just as well.

5. The "Fake Data" Boost

To make the apprentice even better, they added 256,000 synthetic (fake) sonar images to the training mix.

The Analogy: Imagine you are learning to drive. You have 10 real driving lessons. To practice more, you get a driving simulator. Even though the simulator isn't real road, it teaches you the rules of the road.
The Result: Mixing real sonar data with this "simulator" data helped the AI get even sharper, proving that you can learn from fake data if the AI is trained the right way.

The Big Takeaway

This paper teaches us a valuable lesson for the future of AI in difficult fields (like medicine or underwater exploration):

You don't always need the biggest, most expensive AI model.

If you have very little data and a very specific, weird environment, a smaller, carefully designed model that learns specifically for that environment will often beat a giant, general-purpose model. It's better to have a specialist who knows the local dialect than a generalist who knows everything but speaks the wrong language.

Mine-JEPA is that specialist, and it's ready to help keep our ships safe by finding mines faster and more accurately than ever before.

1. Problem Statement

The paper addresses the critical challenge of Side-Scan Sonar (SSS) mine classification, a maritime vision task essential for naval mine countermeasures. The field faces two primary hurdles:

Extreme Data Scarcity: Labeled sonar data is extremely limited (the benchmark used contains only 1,170 images), making standard supervised deep learning difficult.
Large Domain Gap: Existing high-performance vision foundation models (e.g., DINOv3) are pretrained on natural RGB images. SSS imagery relies on acoustic highlight-shadow structures and seabed textures, which differ fundamentally from natural images. Applying models trained on natural images directly to sonar data often leads to suboptimal performance or requires massive amounts of labeled data for fine-tuning.

The authors investigate whether In-Domain Self-Supervised Learning (SSL) can bridge this gap, specifically asking if a model pretrained on a small set of unlabeled sonar images can outperform massive foundation models pretrained on billions of natural images.

2. Methodology: Mine-JEPA

The authors propose Mine-JEPA, a three-stage pipeline designed for data-scarce regimes.

A. Pipeline Overview

Data Preparation: Unlabeled $96 \times 96$ patches are extracted from 1,170 SSS images using a sliding window, generating ~153K unlabeled patches.
In-Domain SSL Pretraining: A Vision Transformer (ViT) backbone is pretrained using the SIGReg loss function on these unlabeled patches.
Probe Evaluation: A lightweight linear or MLP head is attached to the pretrained backbone to evaluate performance on labeled patches (Background, Mine-like Contact, Non-mine Bottom Object).

B. Core Components

SIGReg Loss (LeJEPA): Instead of complex teacher-student architectures (like DINO or BYOL) or contrastive methods (SimCLR), Mine-JEPA uses Sketched Isotropic Gaussian Regularization (SIGReg).
- Mechanism: It combines an invariance term (encouraging augmented views of the same patch to have similar embeddings) and a regularization term (forcing the embedding distribution to match a standard normal distribution via random slicing).
- Advantage: It avoids negative pairs, momentum encoders, and complex heuristics, making it stable and computationally efficient for small batch sizes and limited data.
- Projection Bottleneck: The authors use an intentionally low projection dimension ( $d=16$ ) to prevent overfitting and act as a structural regularizer.
SSS-Adapted Augmentation: Standard natural-image augmentations (color jitter, hue/saturation changes, solarization) are harmful to sonar data. The authors replace these with:
- Brightness and contrast adjustments only (no color/saturation).
- Vertical flips and random rotations ( $\pm 15^\circ$ ) to account for sonar scanning geometry.
- Removal of grayscale conversion and solarization.
Initialization Strategy:
- ImageNet-1K (IN1K): The ViT backbone is initialized with IN1K weights. This provides useful low-level priors (edges, textures) without the rigid, specialized representations of massive foundation models.
- Random Initialization: The projection head is always randomly initialized.
Data Composition (Real+Syn): The authors explore a "Real+Syn" setting, mixing the 153K real patches with ~256K synthetic sonar patches. Despite the modality gap (Real is RGB, Synthetic is grayscale), the heterogeneous data improves performance when paired with the correct SSL objective.

3. Key Contributions

First In-Domain SSL Pipeline for SSS: Mine-JEPA is the first framework to apply in-domain SSL specifically for mine classification in side-scan sonar.
Superiority over Large Foundation Models: The paper demonstrates that a model pretrained on only 1,170 unlabeled images can outperform DINOv3 (pretrained on 1.7 billion images) when the domain gap is significant.
Discovery of Foundation Model Degradation: A critical finding is that applying in-domain SSL to already strong foundation models (like DINOv3) degrades performance by 10–13 percentage points. This suggests that highly specialized representations are brittle and cannot be easily adapted to small, mismatched domains.
Efficiency: The approach achieves competitive results using ViT-Tiny, which has 4 $\times$ fewer parameters than DINOv3, making it viable for resource-constrained platforms like Autonomous Underwater Vehicles (AUVs).

4. Experimental Results

The evaluation was conducted on a public benchmark with 3 classes (Background, MILCO, NOMBO) and a binary setting (Mine vs. Non-mine).

Metric	Mine-JEPA (ViT-Tiny)	DINOv3 (Fine-tuned)	Improvement
3-Class Macro-F1	0.820	0.810	+1.0%
Binary F1 Score	0.935	0.922	+1.3%
MILCO Recall	91.4%	88.1%	+3.3%
Parameters	5.5M	21.5M	75% Reduction

Ablation on Initialization:
- IN1K + SIGReg: Achieved 0.820 F1.
- DINOv3 + SIGReg: Performance dropped to 0.706 (Real data) and 0.677 (Real+Syn), confirming that strong pretraining can be detrimental in this specific small-data regime.
Ablation on Augmentation: Using standard natural-image augmentations on sonar data resulted in a catastrophic drop to 0.312 F1, proving that domain-specific augmentation is a prerequisite, not an optional detail.
SSL Method Comparison: Regularization-based methods (SIGReg, VICReg) benefited from synthetic data, while contrastive (SimCLR) and distillation-based (BYOL) methods degraded or stagnated.

5. Significance and Conclusion

The paper fundamentally shifts the paradigm for maritime computer vision:

Quality over Quantity: In data-scarce, high-domain-gap scenarios, a carefully designed in-domain SSL pipeline with moderate initialization (ImageNet) outperforms massive foundation models.
The "Sweet Spot" of Pretraining: There is a "Goldilocks" zone for initialization. Starting from scratch is too hard; starting from a massive foundation model is too rigid. Starting from a general-purpose model (ImageNet) and adapting it via simple, stable SSL (SIGReg) yields the best results.
Practical Deployment: The success of ViT-Tiny suggests that lightweight, onboard AI for AUVs is feasible without relying on massive compute resources or cloud-based processing.

Limitations: The study relies on a single public dataset (1,170 images) and synthetic data. Future work needs to validate these findings across diverse sonar systems (SAS, FLS) and operational conditions.

In summary, Mine-JEPA proves that for specialized, data-scarce domains like underwater mine detection, domain-matched self-supervised learning is a more effective strategy than simply scaling up foundation models trained on natural images.