A Two-Stage Detection-Tracking Framework for Stable Apple Quality Inspection in Dense Conveyor-Belt Environments

This paper presents a two-stage detection-tracking framework combining YOLOv8 localization, ByteTrack identity maintenance, and ResNet18 defect classification with track-level aggregation to achieve stable, temporally consistent apple quality inspection in dense conveyor-belt environments, outperforming traditional frame-wise inference methods.

The Gist

Imagine a busy apple factory where thousands of apples are rushing down a conveyor belt, bumping into each other, spinning, and sometimes getting covered in shadows or blurry from the speed. The factory needs to sort them: Good apples go to the "Premium" bin, and Bad apples (bruised, rotten, or scabby) go to the "Compost" bin.

The problem? If you just take a quick snapshot of an apple as it zooms by, your eyes (or a computer camera) might get confused. One second, a bruise looks like a shadow; the next second, the apple spins and looks perfect. If the computer makes a decision based on just one snapshot, it might flip-flop: "Good! Bad! Good! Bad!" This leads to chaos and wasted fruit.

This paper proposes a smarter way to sort apples, acting like a two-step detective team that doesn't just look at a single photo, but follows the apple's whole journey.

The Two-Step Team

Step 1: The Spotter (YOLOv8)
Think of this as a highly trained security guard who has spent years watching apples in a sunny orchard. Even though the factory belt looks different (industrial lights, crowded apples), this guard is so good at spotting apples that they can find them instantly in the chaotic factory video. They draw a box around every apple they see.

Step 2: The Tracker (ByteTrack)
Here is the magic trick. In a normal system, the computer might lose track of an apple for a split second and think it's a new apple. This system uses a "Tracker" who acts like a name tag.

• When the Spotter sees an apple, the Tracker gives it a unique ID (like "Apple #42").
• Even if Apple #42 gets partially hidden by another apple or blurs from motion, the Tracker remembers, "That's still Apple #42," and follows it down the line.

The Quality Inspector (ResNet18)

Once the Tracker has followed an apple for a few seconds, a specialized Quality Inspector (a smart AI brain) looks at the apple's skin.

• The Old Way: The Inspector would shout a verdict every time the apple passed a camera: "Bad!" then "Good!" then "Bad!" based on a single, shaky frame.
• The New Way: Because the Tracker has been following Apple #42 for a while, the Inspector gets to see the apple from many angles and under different lights. Instead of shouting immediately, the Inspector waits and collects opinions.

The "Majority Vote" (Aggregation)

This is the most important part. Imagine Apple #42 passes the camera 20 times.

• 18 times, the AI says: "This apple is Good."
• 2 times, the AI gets confused by a shadow and says: "This apple is Bad."

If you just looked at those 2 moments of confusion, you'd throw away a good apple. But this system uses a Majority Vote. It looks at all 20 opinions and says, "Okay, 18 out of 20 say it's good. The final verdict is Good."

This stops the system from panicking over a single blurry frame. It stabilizes the decision, just like how you wouldn't decide a movie is bad because of one bad scene; you wait until the end to judge the whole story.

Why This Matters

The researchers tested this on real factory videos. They found that:

1. Less Confusion: The system stopped flipping between "Good" and "Bad" for the same apple.
2. Better Sorting: By waiting to see the whole "story" of the apple's movement, they could accurately count how many bad apples were actually on the belt, rather than getting a messy, inconsistent count.

In a nutshell: Instead of judging a book by its cover (or an apple by a single blurry photo), this system reads the whole chapter. It follows the apple, gathers enough evidence, and makes a calm, stable decision, ensuring that only the truly bad apples get tossed out.

Technical Summary

Based on the provided manuscript, here is a detailed technical summary of the paper "A Two-Stage Detection–Tracking Framework for Stable Apple Quality Inspection in Dense Conveyor-Belt Environments."

1. Problem Statement

The paper addresses a critical gap in automated fruit inspection systems: the lack of temporal stability in video-based quality assessment.

• Context: Industrial conveyor-belt environments involve densely packed, continuously moving apples.
• Limitations of Current Approaches: Most existing deep learning solutions operate at the image level (frame-wise inference). They treat each video frame independently, ignoring object continuity.
• Consequences: Frame-wise classification is prone to prediction oscillation due to motion blur, partial occlusion, and illumination variations. This leads to unreliable grading outcomes where a single apple might be classified as "defective" in one frame and "healthy" in the next, making industrial deployment impractical.
• Domain Shift: Models trained on orchard data often degrade when deployed in industrial settings due to differences in lighting, background, and object arrangement.

2. Methodology

The authors propose a Two-Stage Detection–Tracking Framework designed to enforce object-level temporal consistency. The pipeline consists of three core components:

A. Apple Detection (Stage 1)

• Model: YOLOv8 (You Only Look Once version 8).
• Training Strategy: The detector is pre-trained on an orchard-based dataset (natural lighting conditions) and deployed directly onto industrial conveyor videos without additional fine-tuning. This tests the model's cross-domain robustness.
• Output: A set of bounding boxes ($D_t$) for each frame $t$.

B. Multi-Object Tracking (Stage 2)

• Algorithm: ByteTrack.
• Function: ByteTrack associates detection boxes across consecutive frames to assign persistent unique Track IDs to each apple instance.
• Advantage: Unlike standard trackers that discard low-confidence detections, ByteTrack utilizes both high- and low-confidence boxes, making it highly robust in dense, occluded scenes typical of conveyor belts. This prevents identity switching and maintains a continuous trajectory ($T_i$) for each apple.

C. Defect Classification & Aggregation

• Classifier: ResNet18, initialized with ImageNet weights and fine-tuned on the Healthy-Defective Fruits dataset (categorized into fresh, bruise, rot, and scab).
• Inference Process: For every tracked apple, the corresponding image region is cropped and passed to the classifier to predict a defect label ($y_t$).
• Temporal Stabilization (Key Innovation): Instead of outputting a decision per frame, the system employs Track-Level Aggregation.
  • All predictions for a specific track ($Y_i$) are collected over the apple's lifespan in the video.
  • Majority Voting is applied to determine the final label ($\hat{y}_i$) for that apple.
  • This mitigates frame-wise noise and ensures a single, stable decision per object.

3. Key Contributions

1. Unified Framework: Integration of detection (YOLOv8), tracking (ByteTrack), and classification (ResNet18) into a cohesive video-based pipeline, moving beyond static image benchmarks.
2. Temporal Consistency Mechanism: Introduction of track-level majority voting to stabilize defect decisions, directly addressing the "prediction oscillation" problem in dynamic environments.
3. Cross-Domain Robustness: Demonstration of an orchard-trained detector functioning effectively in industrial conveyor settings without retraining, highlighting the potential for transfer learning in agriculture.
4. New Evaluation Metrics: Definition of video-level industrial metrics to replace standard image-level accuracy:
   • Track-Level Defect Ratio: The proportion of tracks identified as defective.
   • Temporal Stability: A metric quantifying the reduction in label changes per track (calculated as $1 - \frac{\text{label changes}}{\text{track length}}$).

4. Expected Results & Findings

As a preliminary feasibility study (using publicly available YouTube conveyor videos for inference due to a lack of frame-level ground truth), the paper hypothesizes and demonstrates:

• Improved Stability: The tracking + aggregation approach significantly reduces prediction fluctuations compared to frame-wise inference.
• Reliable Grading: The system produces consistent defect ratios at the video level, which is crucial for industrial quality control.
• Domain Adaptation: The YOLOv8 detector maintains high precision despite the domain shift from orchard to conveyor environments.
• Noise Reduction: The ResNet18 classifier, while accurate on static images, benefits immensely from the temporal aggregation, smoothing out errors caused by motion blur and occlusion.

5. Significance

This work bridges the gap between academic computer vision research and practical industrial deployment.

• Practical Impact: It provides a blueprint for building reliable automated grading systems that do not fail due to the inherent noise of moving conveyor belts.
• Methodological Shift: It argues that for continuous industrial processes, temporal modeling (tracking) is not optional but essential for accurate quality inspection.
• Future Directions: The framework sets the stage for future work in domain generalization, multi-camera integration, and large-scale industrial validation.

In summary, the paper argues that for dense, dynamic industrial environments, tracking is the key to stability. By shifting the decision-making unit from "frame" to "object track," the proposed framework achieves the reliability required for real-world apple quality inspection.