Deep Feature Optimization for Enhanced Fish Freshness Assessment

Technical Summary: Deep Feature Optimization for Enhanced Fish Freshness Assessment

Problem Statement
Assessing fish freshness is critical for food safety and economic stability in the seafood industry. Traditional sensory evaluation methods are subjective, time-consuming, and difficult to scale. While recent deep learning approaches have automated visual freshness prediction, challenges remain regarding accuracy and feature transparency. Specifically, on the public "Freshness of the Fish Eyes" (FFE) dataset, existing deep models have achieved only moderate accuracies (ranging from 63% to 77%), suggesting that distinguishing subtle inter-class differences in eye appearance requires more discriminative and interpretable representations. Furthermore, existing studies often evaluate deep networks in an end-to-end manner without explicitly analyzing how feature abstraction levels or selection mechanisms impact performance.

Methodology
The study proposes a unified, three-stage framework designed to refine and leverage deep visual representations for fish freshness classification. The methodology proceeds as follows:

1. Deep Backbone Fine-Tuning: Five state-of-the-art vision architectures—ResNet-50, DenseNet-121, EfficientNet-B0, ConvNeXt-Base, and Swin-Tiny—were fine-tuned using Random search for hyperparameter optimization. This established a robust baseline for image-based freshness classification.
2. Multi-Level Feature Extraction and Hybrid Classification: Instead of relying solely on the final softmax classification layer, deep embeddings were extracted from intermediate (mid-level) and final (high-level) abstraction stages of each backbone. These features were condensed via Global Average Pooling (GAP) and fed into seven classical machine learning classifiers: Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Artificial Neural Networks, Random Forest, Extra Trees, LightGBM, and CatBoost. This hybrid approach aimed to determine if explicit decision mechanisms could enhance discriminative power beyond standard deep network outputs.
3. Embedded Feature Selection: To address high dimensionality and redundancy, three embedded feature selection methods were applied to identify compact, informative subsets: Light Gradient Boosting Machine (LGBM), Random Forest (RF), and Lasso regression (L1). These methods integrated feature relevance assessment directly into the model training process.
4. Interpretability: Grad-CAM (Gradient-weighted Class Activation Mapping) was employed to visualize the spatial attention of the models, validating that decisions were based on meaningful visual cues related to fish freshness (e.g., eye cloudiness and structure).

Key Contributions

• Systematic Analysis of Abstraction Levels: The study is the first to systematically analyze how deep feature abstraction levels (mid-level vs. high-level) jointly affect fish freshness classification on the FFE dataset.
• Hybrid Deep-ML Framework: It demonstrates a novel integration where deep backbones serve as feature extractors for classical machine learning classifiers, rather than relying on end-to-end deep learning.
• Feature Optimization: The research identifies that embedded feature selection, particularly using LGBM, can significantly reduce feature dimensionality while improving or maintaining accuracy.
• Comprehensive Benchmarking: The study provides a comparative evaluation of five diverse vision architectures (including Transformers and modern ConvNets) and multiple ML classifiers on the same dataset under identical conditions.

Experimental Results
Experiments were conducted on the FFE dataset, comprising 4,390 images across three freshness categories ("Highly Fresh," "Fresh," "Not Fresh").

• Baseline Performance: Among the fine-tuned deep models, Swin-Tiny achieved the highest accuracy of 84.85%, followed closely by ConvNeXt-Base (84.51%).
• Hybrid Performance: Using deep features extracted from the high-level stages (Stage 4) of Swin-Tiny combined with an Extra Trees (ET) classifier, the system achieved an accuracy of 85.88%, surpassing the standalone Swin-Tiny model.
• Feature Selection Impact: Applying LGBM-based feature selection further optimized the results. The best configuration utilized features from Swin-Tiny (reduced to 77 features, representing a 90% reduction from the full 768-dimensional set) classified by a Random Forest (RF) model, achieving a peak accuracy of 85.99%.
• Comparison with State-of-the-Art: The proposed best configuration outperformed recent studies on the same FFE dataset by significant margins: 8.43% over Hoang et al. (2025), 8.69% over Yildiz et al. (2024), and 22.78% over Prasetyo et al. (2022b).
• Efficiency Analysis: While ConvNeXt-Base offered high accuracy, it was computationally intensive. Swin-Tiny emerged as the most balanced solution, offering the highest accuracy with moderate resource requirements (27.5M parameters, 44.1s per epoch).

Significance and Claims
The authors claim that their framework effectively addresses the limitations of previous studies, which often relied on small datasets, single anatomical regions, or exclusively handcrafted features. By systematically combining modern vision transformers with traditional ensemble classifiers and boosting-based feature selection, the study demonstrates a strategy that is both accurate and generalizable for visual quality evaluation.

The paper emphasizes that the improvement stems from the ability of the hybrid framework to capture both semantic patterns and physical cues, while the feature selection step isolates compact, informative subsets that reduce noise. The authors note that while the results are promising, the framework is currently limited by the scale and idealized conditions of the FFE dataset. They suggest that future work should focus on developing hybrid representations that combine deep and handcrafted features to capture both semantic and physical attributes, and validating the approach on larger, more diverse real-world datasets to improve robustness in industrial settings.