A Novel Public Dataset for Strawberry (Fragaria x ananassa) Ripeness Detection and Comparative Evaluation of YOLO-Based Models

This study introduces a novel public dataset of 566 strawberry images captured under variable conditions in Turkey to address data scarcity and evaluates YOLOv8, YOLOv9, and YOLO11 models for ripeness detection, finding that YOLOv8s achieves the best overall mAP@50 performance of 86.09% while demonstrating the efficiency of small and medium-sized models for smart agriculture applications.

The Gist

Imagine you are a strawberry farmer. Your job is to pick the perfect berries at the exact moment they are ready to eat. If you pick them too early, they are sour and pale. If you wait too long, they turn into mushy, bruised messes.

For centuries, farmers have done this by looking at the berries with their own eyes. But human eyes get tired, lighting changes, and one person might think a berry is "ready" while another thinks it's "too green." It's subjective, slow, and expensive.

This paper is about teaching a computer to be the perfect, tireless strawberry picker. Here is the story of how they did it, explained simply.

1. The Missing Puzzle Piece: The Dataset

Before you can teach a computer to recognize anything, you need to show it thousands of examples. In the world of AI, this is called a dataset.

The problem? Most researchers kept their strawberry photos secret, like a chef guarding a secret recipe. This made it impossible to compare who was actually the best at teaching computers.

The Solution: The authors of this paper decided to be the "open-source chefs." They went into two different greenhouses in Turkey, took 566 photos of strawberries under all kinds of lighting (bright sun, shady corners, cloudy days), and labeled them by hand.

• They marked the Unripe (green/white) ones.
• They marked the Semi-ripe (pinkish) ones.
• They marked the Fully-ripe (bright red) ones.

They put all this data on a public website (Kaggle) so anyone in the world could download it and use it. This is their biggest gift to the scientific community.

2. The Race: The YOLO Family

To find the best computer brain for this job, they didn't just pick one. They organized a race between three generations of a famous AI family called YOLO (which stands for "You Only Look Once"). Think of these models as different types of workers:

• YOLOv8: The reliable veteran. It's been around a while and is very steady.
• YOLOv9: The new kid with a fancy trick. It uses a special method to remember details better so it doesn't lose information as it gets deeper into the image.
• YOLO11: The latest, cutting-edge model. It's designed to be super efficient and fast.

But here's the twist: Each of these models comes in different sizes, like cars.

• Nano/Small: Like a compact city car. Fast, cheap on gas (computing power), but maybe not the most powerful.
• Medium/Large: Like a family SUV. More powerful, but heavier and uses more gas.
• Extra-Large: Like a massive semi-truck. Huge power, but very heavy and expensive to run.

3. The Results: Bigger Isn't Always Better

The researchers ran a marathon with all these models on their new strawberry dataset. Here is what they found:

• The "Goldilocks" Zone: The biggest, heaviest models (the semi-trucks) did not win. In fact, they sometimes performed worse than the smaller ones. Why? Because the dataset wasn't big enough to teach these massive brains everything they needed to know. They got confused and started "overthinking" (a technical term called overfitting).
• The Winners: The Small and Medium models (the compact cars) were the champions.
  • YOLOv8s (Small): Won the overall race. It had the best balance of speed and accuracy. It was the most reliable "all-rounder."
  • YOLOv9c (Medium): Was the most precise. It rarely made mistakes saying a berry was ripe when it wasn't. It was very careful.
  • YOLO11s (Small): Was the most sensitive. It was great at spotting the tricky "semi-ripe" berries that are hard to see, though it sometimes got a little too excited and saw ripe berries where there were none.

4. The Big Lesson

The most important takeaway from this paper is a lesson for life: Bigger isn't always better.

In the past, people thought, "If I build a bigger, more complex computer brain, it will definitely be smarter." This study proved that wrong. For a specific job like picking strawberries in a greenhouse, a smaller, well-tuned brain works better than a giant, over-complicated one. It's like using a Swiss Army knife instead of a chainsaw to cut a sandwich.

5. Why This Matters

This research helps move us closer to Smart Farming.

• Imagine a robot arm in a greenhouse that can look at a strawberry, decide if it's ready, and pick it without hurting it.
• This paper gives us the map (the public dataset) and the best tools (the small YOLO models) to build those robots.

In a nutshell: The authors gave the world a free, high-quality photo album of strawberries and proved that for teaching computers to pick fruit, you don't need a super-computer; you just need the right-sized tool for the job.

Technical Summary

1. Problem Statement

Strawberry ripeness determination is critical for minimizing post-harvest losses and ensuring product quality. However, traditional visual assessment methods are subjective, labor-intensive, and prone to human error, especially under variable lighting conditions in greenhouses. While deep learning (DL) offers a solution, the field faces three major challenges:

• Lack of Public Data: Most existing studies rely on private datasets, hindering reproducibility and fair comparison between different algorithms.
• Inconsistent Evaluation: Studies often use different architectures and experimental conditions, making it difficult to determine which model is truly superior.
• Efficiency vs. Accuracy Trade-off: There is a lack of systematic analysis regarding whether larger, more complex models actually yield better performance on agricultural datasets, or if smaller, more efficient models are sufficient.

2. Methodology

The study proposes an end-to-end deep learning workflow comprising dataset creation, model selection, and comprehensive evaluation.

A. Dataset Creation

• Source: Images were collected from two different greenhouses in Kayseri, Türkiye, using Samsung NX cameras and mobile phones to simulate varied imaging conditions.
• Conditions: Data was captured under diverse natural lighting scenarios (direct sunlight, partial shading, diffuse light) and varying fruit densities.
• Composition:
  • Total Images: 566 (465 for training, 101 for testing).
  • Annotations: 1,734 bounding boxes labeled manually.
  • Classes: Three ripeness stages: Fully-ripe (520 instances), Semi-ripe (247 instances), and Unripe (967 instances).
• Availability: The dataset is publicly available on Kaggle to ensure reproducibility.

B. Model Architecture

The study comparatively evaluates three generations of the YOLO (You Only Look Once) object detection family: YOLOv8, YOLOv9, and YOLO11.

• Versions Tested: Nano (n), Small (s), Medium (m), Large (l), and Extra-large (x) variants for each generation.
• Architectural Features:
  • YOLOv8: Uses C2f blocks and an anchor-free detection head.
  • YOLOv9: Introduces RepNCSPELAN4 blocks and CBFuse modules to reduce information loss in deep layers.
  • YOLO11: Utilizes optimized C3k2 blocks and C2PSA (Parallel Spatial Attention) modules for better feature extraction.

C. Evaluation Metrics

Performance was assessed using a dual-axis approach:

1. Accuracy Metrics: Precision (P), Recall (R), and Mean Average Precision at IoU 0.5 (mAP@50).
2. Efficiency Metrics: Floating Point Operations (FLOPs), Number of Parameters (Params), and Frames Per Second (FPS).

3. Key Contributions

1. Novel Public Dataset: Introduction of the first publicly available, multi-class strawberry ripeness dataset collected from two distinct greenhouse environments with variable lighting, addressing the scarcity of open-source agricultural data.
2. Systematic Comparative Analysis: A rigorous, apples-to-apples comparison of YOLOv8, v9, and v11 across all scale variants (n, s, m, l, x) under identical experimental conditions.
3. Scale-Performance Insights: The study challenges the assumption that "larger models equal better performance," providing empirical evidence on the non-linear relationship between model complexity and detection accuracy in agricultural contexts.

4. Key Results

Performance Metrics

• Best Overall Accuracy (mAP@50): YOLOv8s achieved the highest score of 86.09%, outperforming all other models despite having fewer parameters than the "x" variants.
• Highest Precision: YOLOv9c achieved 90.94%, indicating a conservative strategy that minimizes false positives (crucial for avoiding premature harvesting).
• Highest Recall: YOLO11s achieved 83.74%, demonstrating superior sensitivity in detecting difficult classes, particularly the morphologically intermediate "semi-ripe" strawberries.

Efficiency vs. Complexity

• Non-Linear Scaling: Increasing model size did not linearly improve accuracy. The largest models (e.g., YOLOv8x, YOLOv9e, YOLO11x) with high computational costs (up to 344.1 GFLOPs) yielded lower mAP scores than smaller models.
• Optimal Range: Models in the 10–12 million parameter range (specifically YOLOv8s and YOLO11s) offered the best balance between accuracy and computational cost.
• Overfitting: The decline in performance for extra-large models suggests that the dataset size (566 images) was insufficient to train high-capacity models without overfitting or sensitivity to noise.

Class-Specific Analysis

• Semi-ripe Detection: This class proved the most challenging due to subtle color transitions. YOLO11s showed the highest sensitivity here (13 True Positives vs. 8 for YOLOv9c), though it occasionally produced false positives in complex backgrounds.
• Fully-ripe/Unripe: All top models performed well on distinct classes, but confidence scores dropped significantly for semi-ripe samples in models like YOLOv8s.

5. Significance and Conclusion

This research provides a fundamental reference for Smart Agriculture and Autonomous Harvesting Systems.

• Practical Implication: For real-time robotic harvesting, the study suggests that deploying massive models is unnecessary and inefficient. Instead, compact models like YOLOv8s or YOLO11s should be prioritized for their optimal balance of speed, accuracy, and hardware requirements.
• Strategic Selection: The choice of model should depend on the specific application goal:
  • Use YOLOv9c if minimizing false positives (avoiding picking unripe fruit) is the priority.
  • Use YOLO11s if maximizing detection of intermediate ripeness stages is critical, even at the cost of slightly more false positives.
• Future Directions: The authors note limitations regarding geographical diversity and class imbalance (dominance of unripe fruit). Future work should focus on expanding datasets to open-field conditions and different cultivars to improve generalizability.

In summary, the paper establishes that architectural efficiency is more critical than raw model size for strawberry ripeness detection and provides a reproducible framework for future agricultural AI research.