Technical Summary: Benchmarking ResNet Backbones in RT-DETR under Environmental Conditions

Problem Statement

In competitive robotics, such as RoboCup and HuroCup, autonomous systems rely heavily on real-time visual perception to detect and track small, geometrically simple objects (e.g., balls) under strict temporal constraints. While transformer-based detectors like RT-DETR have emerged as powerful alternatives to convolutional architectures by enabling end-to-end detection without Non-Maximum Suppression (NMS), there is a lack of literature regarding how backbone scale and environmental variations specifically impact their performance in these settings.

Existing frameworks often struggle with environmental variability, particularly illumination changes (uneven lighting, flickering, natural sunlight) and background contrast issues (e.g., gray carpets, wooden walls), which can lead to color misclassification and detector instability. Furthermore, the interaction between backbone depth, dropout regularization, and these environmental factors in real-time transformer detectors remains under-explored. This work addresses the need to determine the optimal balance between detection accuracy, confidence, and inference latency when deploying RT-DETR under such variable conditions.

Methodology

The study presents a controlled comparative evaluation of the RT-DETR architecture using four ResNet backbone variants: ResNet18, ResNet34, ResNet50, and ResNet101.

Experimental Setup

• Dataset: A custom dataset aggregating 8,806 images from multiple Roboflow repositories, featuring spherical objects (blue, green, orange, and white balls). The red ball class was excluded due to chromatic similarity with orange under artificial lighting.
• Preprocessing: Images were resized to 640×640 pixels. Static data augmentation included random rotations (±13°) to improve orientation invariance while preserving spherical geometry.
• Training Configuration: All models were trained from scratch for 150 epochs on an NVIDIA RTX 4070 Ti GPU using the Ultralytics framework with Stochastic Gradient Descent (SGD).
• Regularization Modes: Each backbone was trained in two modes:
  1. Standard Mode: No dropout (Dropout rate = 0.0).
  2. Regularized Mode: Dropout rate of 0.2 applied to the transformer encoder's attention mechanisms.
• Evaluation Scenarios: Inference was tested on video sequences under four distinct conditions:
  1. Full Light (Artificial) vs. Low Light.
  2. White Background vs. Black Background.
• Metrics: Performance was evaluated using Average Confidence (mean prediction score), Inference Time (latency in ms/frame), and Classification Accuracy (correct detections per class).

Key Contributions

1. Systematic Backbone Evaluation: A comparative analysis of RT-DETR with ResNet backbones ranging from ResNet18 to ResNet101 to quantify the trade-offs between representational depth, accuracy, and latency.
2. Environmental Robustness Analysis: A controlled assessment of how illumination changes and background contrast variations affect detector performance, isolating these factors from architectural differences.
3. Regularization Impact: An evaluation of dropout (0.2) on RT-DETR to determine its effect on prediction confidence and classification stability under varying operating conditions.

Experimental Results

Impact of Illumination

• Confidence vs. Accuracy: Lighting variations significantly impacted prediction confidence but had minimal effect on classification accuracy, which remained consistently high (often approaching or exceeding 1.00) across most configurations.
• Backbone Performance:
  • ResNet50 demonstrated the best overall trade-off under illumination variation, achieving near-perfect accuracy, confidence values up to ~0.869, and latency around 0.058–0.059 ms.
  • ResNet101 did not show clear improvements over ResNet50, suggesting diminishing returns with increased depth for this specific task.
  • ResNet18 offered lower computational costs but exhibited slightly reduced confidence under challenging lighting.
• Regularization: Applying dropout (0.2) generally stabilized confidence values across lighting conditions, reducing the gap between Full Light and Low Light performance, though it did not eliminate sensitivity to extreme conditions.

Impact of Background Contrast

• Confidence Sensitivity: Background changes (White vs. Black) affected confidence more than accuracy. Under standard conditions (Dropout 0.0), confidence values were more dispersed, particularly for blue and white balls against contrasting backgrounds.
• Regularization Effect: Dropout (0.2) homogenized confidence values and reduced the performance gap between white and black backgrounds, improving the robustness of internal feature representations.
• Backbone Performance:
  • Under background variation, ResNet34 provided the most balanced performance, achieving near-perfect accuracy and the highest confidence values (up to ~0.887) while maintaining low latency and a compact model size.

Latency

Inference latency remained largely unaffected by environmental conditions (lighting or background) and was primarily determined by the backbone architecture's depth.

Significance and Conclusions

The paper concludes that environmental factors (lighting and contrast) primarily degrade the reliability (confidence) of predictions rather than their correctness (accuracy) or computational cost. While deeper models (ResNet101) do not guarantee superior robustness, intermediate-depth models offer the most stable behavior for competitive robotics tasks.

• Optimal Architectures: The study identifies that the optimal architecture is context-dependent. ResNet50 is recommended for scenarios dominated by illumination variations, while ResNet34 is superior for environments with significant background contrast variations.
• Role of Regularization: Dropout improves stability and reduces variability in confidence across conditions but does not fully compensate for difficult visual environments.
• Design Implication: For real-world robotic applications involving simple geometric objects, intermediate architectures combined with appropriate regularization provide a more favorable balance between performance, efficiency, and robustness than simply scaling up model depth.