Non-invasive Growth Monitoring of Small Freshwater Fish in Home Aquariums via Stereo Vision

This paper proposes a non-invasive, refraction-aware stereo vision system that utilizes a YOLOv11-Pose network and specialized epipolar constraints to accurately estimate the length of small freshwater fish in home aquariums, validated on a new dataset of endangered Sulawesi ricefish.

The Gist

Imagine you have a tiny, shy pet fish in your home aquarium. You love watching it, but you also want to know if it's growing healthy and strong. The old-fashioned way to check its size? You'd have to catch it, pull it out of the water, and measure it with a ruler. But that's stressful for the fish, and if you do it too often, you might hurt it.

This paper introduces a digital "magic ruler" that measures fish without ever touching them. It uses a special camera setup and some clever computer tricks to figure out exactly how long a fish is, even when it's swimming fast, hiding behind plants, or when the water distorts the view.

Here is how the system works, broken down into simple steps:

1. The "Two-Eyed" Camera (Stereo Vision)

Think of how you have two eyes. Your brain combines what both eyes see to understand how far away things are (depth). This system uses two cameras side-by-side, acting like a pair of eyes.

• The Problem: When you look through a fish tank, the glass and water bend the light (refraction). It's like looking through a funhouse mirror; straight lines look curved. Standard computer vision tools get confused by this and can't measure distances accurately.
• The Fix: The researchers taught the computer to understand the "bending" rules of the glass and water. They created a special map (called epipolar curves) that tells the computer exactly where a fish seen in the left camera should appear in the right camera, even if the image is warped.

2. The "Super Detective" AI (YOLOv11-Pose)

Once the cameras are set up, the system needs to find the fish. It uses an AI called YOLOv11-Pose.

• What it does: Imagine a super-fast detective scanning the tank. It doesn't just draw a box around the fish; it pinpoints specific body parts like the mouth, eyes, fins, and tail.
• The "Quality Score": Sometimes the fish is blurry, hiding behind a plant, or swimming straight toward the camera (making it look like a tiny dot). The AI has a special "honesty check." It assigns a quality score to every fish it sees:
  • High Quality: "I can clearly see the mouth and tail. I'm confident!"
  • Low Quality: "It's too blurry or hidden. I shouldn't guess."
  • Why this matters: If the computer guesses the size of a blurry fish, it ruins the data. So, the system automatically throws away the "low quality" guesses, just like you would ignore a blurry photo when trying to measure something.

3. The "3D Puzzle Solver" (Triangulation)

Once the AI finds a fish in both camera views and confirms the image is clear, it plays a game of 3D puzzle.

• It takes the "mouth" point from the left camera and the "mouth" point from the right camera.
• Using the special "bending light" math, it calculates exactly where that mouth is floating in 3D space.
• It does the same for the tail.
• Finally, it measures the distance between the 3D mouth and the 3D tail. Bam! You have the fish's length.

4. The "Swimming Direction" Filter

The system is smart enough to know when a fish is trying to trick it.

• If a fish swims directly toward the camera, its body looks like a small circle. The AI can't tell how long it is.
• The system checks the angle: "Is this fish swimming at me?" If yes, it ignores that measurement. It only measures fish swimming sideways, where the length is visible.

5. The "Template" Trick (Refining the Details)

To get super precise measurements, the system tries to "zoom in" on the fish's features.

• It takes a tiny square image of the fish's eye from the left camera and tries to find that exact same square in the right camera.
• The Catch: This works great if the background is plain (like a white wall). But if the background is full of plants and rocks, the computer gets confused, thinking a leaf is the fish's eye.
• The Result: The researchers found that this "zoom-in" trick helps accuracy in simple tanks but can actually make things worse in messy, plant-filled tanks. So, the system adapts based on the environment.

Why This Matters

This technology is a game-changer for:

• Home Aquarium Owners: You can keep a log of your fish's growth without ever stressing them out.
• Conservationists: They can monitor endangered species (like the tiny Sulawesi ricefish used in this study) to see if they are healthy or sick.
• Fish Farms: It helps farmers spot sick fish early by noticing if their growth has stalled.

In a nutshell: This paper teaches computers how to be patient, honest, and mathematically smart enough to measure tiny, wiggly fish through glass and water, all without the fish ever knowing they are being measured. It turns a messy, distorted underwater world into precise, reliable data.

Technical Summary

1. Problem Statement

Monitoring fish growth is critical for aquaculture and home aquariums to detect health issues, stress, or nutritional deficiencies. However, traditional manual measurement is invasive, time-consuming, and stressful for the fish.

• Challenges in Aquariums:
  • Refraction: The air-glass-water interface violates standard pinhole camera assumptions, causing epipolar lines to become curves and breaking standard stereo vision libraries.
  • Target Characteristics: The target species (Sulawesi ricefish) are small (up to 80mm), partially transparent, fast-moving, and often hide behind plants or decor.
  • Visual Artifacts: Motion blur, low contrast, and variable lighting make keypoint detection difficult.
  • Existing Limitations: Previous methods often require controlled environments (white boxes), single-camera setups with fixed distances, or fail to account for depth variations and refraction, leading to inaccurate length estimations.

2. Methodology

The authors propose a refraction-aware stereo vision system that combines deep learning with physical optics modeling. The pipeline consists of four main stages:

A. Dataset Creation

• Subject: Endangered Sulawesi ricefish (Oryzias celebensis).
• Hardware: Stereo setup with two Basler a2A2448-75ucPRO cameras (6mm lenses).
• Data: 104 images with 4,331 annotated fish instances.
• Annotations: Each fish has a bounding box, five anatomical keypoints (mouth, eye, dorsal fin, ventral fin, caudal fin), and a quality label (High, Medium, Low) based on visibility.
• Scenarios: Two test scenes were created: one with a natural background (plants/decor) and one with a white paper background to simulate home aquariums against a wall.

B. Deep Learning Model (YOLOv11-Pose)

• Architecture: Based on YOLOv11-Pose, extended with an additional quality estimation head.
• Function:
  1. Detects fish bounding boxes.
  2. Predicts 5 anatomical keypoints.
  3. Classifies detection quality (High/Medium/Low) to filter unreliable data.
• Training Strategy: Two-stage training. First, the backbone and localization heads are trained. Second, the backbone is frozen, and only the quality head is optimized to prevent overfitting and ensure it learns reliability rather than memorizing samples.

C. Stereo Matching & Refraction Modeling

• Epipolar Curves: Instead of linear epipolar lines, the system uses epipolar curves derived from the axial camera model to account for air-glass-water refraction.
• Cost Function: A greedy assignment strategy matches fish between stereo pairs using a cost function $L(i, j)$ combining:
  1. Epipolar Distance ($L_p$): Distance to the epipolar curve.
  2. Size Similarity ($L_s$): Difference in bounding box dimensions.
  3. Keypoint Configuration ($L_k$): Relative distances between keypoints.
• Keypoint Refinement: A template matching approach (21x21 pixels) refines keypoint positions, constrained to the epipolar curve and a $\pm30$ pixel search window.

D. Filtering Pipeline

To ensure accuracy, the system applies strict filters before 3D triangulation:

1. Quality Filter: Discards "Low" and "Medium" quality detections.
2. Orientation Filter:
   • Removes fish with an aspect ratio < 1.5 (too square/obscured).
   • Removes fish swimming directly toward or away from the camera (angle < 45°), as keypoints are hard to localize in these poses.

E. 3D Reconstruction

• Uses refraction-aware 3D triangulation to calculate the 3D coordinates of the matched keypoints.
• Fish Length: Computed as the Euclidean distance between the 3D mouth and caudal fin keypoints.

3. Key Contributions

1. Non-Invasive System: A practical stereo-vision solution for home aquariums that avoids handling fish.
2. Refraction-Aware Stereo: Explicitly models the air-glass-water interface using epipolar curves, enabling robust matching where standard libraries fail.
3. Learned Quality Assessment: Introduces a novel YOLO head to predict detection reliability, which is proven essential for filtering out noise in complex environments.
4. New Dataset: Release of a stereo dataset of small, transparent freshwater fish with 3D localization and quality annotations.

4. Results

The system was evaluated on the Sulawesi ricefish dataset using Root Mean Squared Error (RMSE) of length estimation.

• Quality Filtering Impact: Filtering low-quality detections reduced RMSE by nearly 50% in some configurations. It was found that false positives (predicting low-quality fish as high-quality) were more detrimental than missing some valid fish.
• Stereo Matching: The combination of the Quality Filter (Qu) and Direction Filter (Di) yielded the best results, reducing bad matches to <2% in optimal configurations.
• Template Matching Nuance:
  • White Background (wBG): Template matching significantly improved accuracy (e.g., reducing RMSE to ~9.22mm with the 'm' backbone).
  • Natural Background (noBG): Template matching often degraded accuracy because background patterns in the template misled the alignment.
• Best Configuration: Using the YOLO 'm' (medium) backbone with Quality and Direction filters achieved the lowest error rates (approx. 6.11mm RMSE on the natural background and 9.22mm on the white background).
• Runtime: The system processes ~5 frame pairs per second on an RTX 6000 GPU. Keypoint refinement via template matching accounts for ~75% of the runtime.

5. Significance and Conclusion

This paper demonstrates that accurate, non-invasive fish growth monitoring is feasible in uncontrolled home aquarium environments by addressing the specific physics of refraction and the visual challenges of small, transparent subjects.

• Practicality: The system is lightweight enough for home users and small organizations.
• Robustness: The integration of learned quality assessment and physical refraction modeling overcomes the limitations of previous methods that required controlled setups.
• Future Work: The authors plan to integrate model-based deblurring to handle motion blur (common in aquariums) and optimize the runtime of the template matching step.

The study highlights that filtering unreliable data is more critical than refining good data, and that background complexity significantly influences the efficacy of template-based refinement strategies.