Automated morphometry and weight prediction of juvenile Chinook Salmon leveraging open-source deep learning models

This paper presents an automated, open-source deep learning framework using the Segment Anything Model (SAM) to predict the weight of juvenile Chinook Salmon from 2D images with high accuracy (R²=0.99), thereby minimizing the need for physical handling of threatened fish populations.

The Gist

Imagine you are a fish biologist trying to check the health of a tiny, endangered baby salmon. Traditionally, to measure its length and guess its weight, you'd have to scoop it out of the water, lay it on a ruler, weigh it on a scale, and then put it back. This is stressful for the fish (like a human getting a check-up while being held down) and takes a long time.

This paper introduces a new, "hands-free" way to do this using a digital camera and a smart computer program. Think of it as giving the fish a "digital check-up" without ever touching it.

Here is how the system works, broken down into simple steps:

1. The Setup: A Fishy Photo Booth

Instead of a net and a scale, the researchers built a simple, inexpensive underwater viewing box. It's like a fishy photo booth.

• The baby salmon swims into a clear chamber filled with water.
• There is a grid on the back (like graph paper) that helps the computer know the size of things.
• A camera snaps a picture of the fish from the side.
• The fish swims right back out, never having left the water or been stressed.

2. The "Magic Eye": Segment Anything Model (SAM)

Once the photo is taken, the computer needs to find the fish in the picture. The researchers used a super-smart, free AI tool called SAM (Segment Anything Model).

• The Analogy: Imagine you have a photo of a messy room with a toy car in it. You draw a rough box around the car with your finger. SAM is like a magical highlighter that instantly knows, "Okay, you want the car," and perfectly traces the outline of the car, ignoring the rest of the room.
• In this case, the user just draws a box around the fish and tells the computer which way the fish is facing. The AI then draws a perfect outline around the fish's body.

3. The "Fin-Trimming" Trick

Fish have fins that stick up, down, or sideways. If you try to guess a fish's weight by looking at its outline, a fin sticking up makes the fish look taller and heavier than it really is.

• The Analogy: Imagine trying to guess the weight of a person wearing a giant, fluffy hat. If you measure them with the hat, you'll think they are huge. You need to take the hat off to get the real measurement.
• The computer automatically finds the fins (tail, back fin, belly fin, etc.) and "erases" them from the outline. It creates a smooth, fin-less shape that represents the actual body of the fish.

4. The "Balloon" Math: Guessing the Weight

Now that the computer has a clean outline of the fish, how does it know the weight?

• The Analogy: Imagine the fish is a 3D balloon. If you take a 2D photo of the balloon, you can see its width and height. If you assume the balloon is round all the way around (like a football), you can mathematically guess how much air is inside (volume). Since fish are mostly water, their volume is directly related to their weight.
• The computer measures the fish's "surface area" (how much skin it has in the photo) and its height. It uses a formula to pretend the fish is an oval ball (an ellipsoid).
• The Result: By calculating the volume of this imaginary 3D ball, the computer can predict the weight with amazing accuracy.

5. The Results: Spot On!

The researchers tested this on 149 baby salmon.

• The Accuracy: The computer's guess was incredibly close to the real weight. On average, it was only off by 0.16 grams (that's less than the weight of a single grape!).
• The Efficiency: It did this without ever stressing the fish out or requiring a human to spend hours measuring every single point on the fish.

Why Does This Matter?

• Saving the Fish: Endangered fish are fragile. This method lets scientists gather critical data without hurting or stressing the animals.
• Speed: It turns a slow, manual process into an automated one.
• Future Potential: Because the software is open-source (free for anyone to use), other scientists can adapt this "fishy photo booth" for other types of fish, helping to protect ecosystems all over the world.

In short, this paper is about teaching a computer to look at a photo of a fish, ignore the wiggly fins, imagine the fish as a 3D balloon, and tell you exactly how much it weighs—all while the fish stays happy and swimming in the water.

Technical Summary

1. Problem Statement

The study addresses two critical challenges in fisheries management, specifically regarding threatened and endangered juvenile Chinook Salmon (Oncorhynchus tshawytscha):

• Fish Handling Stress: Traditional methods for measuring morphometric data (length, weight, condition factors) require removing fish from water, handling them physically, and often anesthetizing them. This causes significant stress and can impact the health and survival of vulnerable populations.
• Data Processing Bottlenecks: Previous automated attempts to digitize fish measurements relied on manual placement of landmark points by trained users. This process is time-consuming, prone to user error, and creates a bottleneck in data processing pipelines, limiting the scale of data collection.

The goal was to develop a system that minimizes fish handling (keeping fish submerged) while automating the extraction of accurate morphometric features to predict weight.

2. Methodology

The authors developed a software framework called HandsFreeFishing, which integrates open-source deep learning with custom geometric algorithms. The pipeline consists of four main stages:

A. Image Acquisition

• Fish are photographed inside a specialized, low-cost "fish viewer" containing a 5x5mm grid for scale calibration.
• Images are taken from a side profile while the fish remains submerged.
• User Input: The user provides a minimal bounding box around the fish and specifies orientation (left/right facing, upright/upside down). No manual landmarking is required.

B. Automated Segmentation (Deep Learning)

• Model: The system utilizes Meta's Segment Anything Model (SAM), an open-source foundation model, to generate initial segmentation masks of the fish from the raw images.
• Contour Processing: The raw mask is refined using Elliptic Fourier Analysis to create a smooth, discretized 2D contour ($c = [c_0, c_1, ..., c_n]$).
• Scale Calibration: A template matching algorithm identifies the 5x5mm grid in the background to convert pixel measurements to millimeters.
• Fin Removal: Since fins can be in various positions (affecting surface area calculations) and have different densities than the body, the system uses heuristics to detect and segment individual fins (dorsal, adipose, caudal, etc.). These fin regions are then masked out to create a "no-fin" segmentation, which better represents the main body volume.
• Feature Extraction: The system also segments the eye to measure eye diameter.

C. Feature Extraction

Once the "no-fin" contour is established, the program calculates:

• Fork Length: Determined by finding the snout tip and the center of the caudal fork (using local maxima detection on the tail contour).
• Surface Area (SA): The 2D projected area of the fish body.
• Best-Fit Ellipse: An ellipse is fitted to the contour to extract semi-axes $a$ (length) and $b$ (height).
• Landmark Points: 16 automated landmark points are placed along the contour to create a "truss lattice" for additional shape analysis.
• Partitioned Surface Areas: The body is divided into sections to allow for non-constant density modeling.

D. Weight Prediction Models

The authors trained several regression models (Linear and Multiple Linear Regression) using Python and Scikit-Learn. The models rely on the biological assumption that fish volume is proportional to weight.

• Geometric Assumptions: The fish body is approximated as a 3D ellipsoid.
  • Volume $V = \frac{4}{3}\pi abc$.
  • Assuming girth ($c$) is proportional to height ($b$), $V \propto SA \times b$.
  • Assuming all proportions are constant ($a \propto b \propto c$), $V \propto SA^{1.5}$.
• Models Tested:
  • Model 1: Uses $V_1 = SA \times b$ (Surface Area $\times$ Height).
  • Model 2: Uses $SA^{1.5}$.
  • Model 3-7: Various combinations of surface area, axes, partitioned areas, and landmark lengths (cubed).

3. Key Contributions

• HandsFreeFishing Program: A fully automated, open-source pipeline (available on GitHub) that reduces user interaction to a simple bounding box and orientation check.
• Integration of SAM: Successfully adapted Meta's Segment Anything Model for specific biological segmentation tasks without extensive retraining, demonstrating the utility of foundation models in ecological monitoring.
• Non-Invasive Workflow: A validated method to measure weight and condition factors without removing fish from water, directly addressing conservation needs for endangered species.
• Robustness to Imperfect Data: The system is designed to handle variable image qualities and segmentation artifacts (e.g., fin movement) through geometric post-processing (convex hulls and largest connected component selection).

4. Results

The system was tested on a dataset of 149 images (fork length 27–90mm, weight 0.31g–7.74g). A cleaned subset of 109 images (excluding poor segmentations) was also analyzed.

• Best Performing Model: Model 1 ($V_1 = SA \times b$) achieved the highest accuracy.
  • R-squared ($R^2$): 0.99 (on the cleaned dataset).
  • Mean Absolute Error (MAE): 0.16 g.
  • Mean Absolute Percentage Error (MAPE): 12%.
  • AIC Score: -19.98.
• Comparison:
  • Model 2 ($SA^{1.5}$) performed similarly but had a slightly higher MAPE (13%).
  • Models using landmark lengths (Model 6 & 7) performed significantly worse, suggesting that global surface area and height are more robust predictors than complex landmark trusses for this specific dataset size.
• Generalization: Even with the full, uncleaned dataset (149 images), Model 1 maintained an $R^2$ of 0.98, demonstrating the system's resilience to segmentation noise.

5. Significance and Future Work

• Conservation Impact: This technology allows researchers to monitor the health and growth of threatened Chinook Salmon populations with minimal stress, enabling more frequent and larger-scale data collection.
• Scalability: The open-source nature of the code and the use of general-purpose deep learning models (SAM) mean the framework can be easily adapted to other fish species and life stages.
• Future Directions: The authors suggest that collecting larger datasets could improve complex models (like Model 5 and 6). Future work may include better anatomical landmarking (e.g., midline segmentation) and predicting other metrics like the Fulton's condition factor ($K$).

In conclusion, the paper demonstrates that combining open-source deep learning with geometric heuristics can replace manual, invasive fish measurement protocols with a highly accurate, automated, and non-invasive alternative.