🤖 AI

Ultralytics YOLO26: Unified Real-Time End-to-End Vision Models

Ultralytics YOLO26 introduces a unified family of real-time end-to-end vision models that eliminates non-maximum suppression and Distribution Focal Loss through a dual-head architecture and advanced training strategies, achieving state-of-the-art accuracy-latency performance across multiple tasks including detection, segmentation, pose estimation, and open-vocabulary inference.

Original authors: Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Fatih Cagatay Akyon, Muhammet Esat Kalfaoglu

Published 2026-06-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Glenn Jocher, Jing Qiu, Mengyu Liu, Shuai Lyu, Fatih Cagatay Akyon, Muhammet Esat Kalfaoglu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-fast security guard whose job is to spot objects in a video stream in real-time. For years, the best guards (called "YOLO" models) have been great, but they had a few annoying habits: they needed a slow "double-check" step after spotting something, they carried heavy backpacks of extra data that slowed them down, and they often missed tiny, distant objects because their training rules were too strict.

The paper introduces YOLO26, a new generation of these vision guards designed to be faster, lighter, and sharper than ever before. Here is how they fixed the old problems, explained with everyday analogies:

1. The "No-Double-Check" Rule (NMS-Free Inference)

The Old Problem: Previously, when a guard spotted a car, they might see it three or four times in slightly different spots. They had to stop and run a "Non-Maximum Suppression" (NMS) process—a slow, manual double-check—to pick just one best guess and throw away the duplicates. This took time.
The YOLO26 Fix: YOLO26 uses a dual-head design. Think of it as having two specialized workers:

Worker A (The Speedster): This worker is trained to pick exactly one perfect guess for every object immediately. No double-checking needed. It's like a sniper who hits the target on the first try.
Worker B (The Maximizer): This worker still looks at everything and can be used if you need the absolute highest accuracy and don't mind the extra "double-check" time.
The Result: You get a system that is "end-to-end," meaning it goes straight from seeing the image to giving the answer without stopping for a second opinion.

2. Ditching the Heavy Backpack (Removing DFL)

The Old Problem: Recent models carried a heavy "Distribution Focal Loss" (DFL) backpack. This backpack tried to predict the size of an object by guessing from a list of 16 different possibilities for every single edge. It made the model heavy (more memory) and slow, especially for tiny models. It also had a "max size" limit; if an object was too big for the list, the model got confused.
The YOLO26 Fix: They threw the backpack away. Instead of guessing from a list, the model now just directly measures the size, like using a ruler instead of guessing from a menu.
The Result: The model is lighter, faster, and can now accurately measure very large objects without hitting a "ceiling" on how big they can be.

3. The "Tiny Object" Safety Net (STAL)

The Old Problem: The old training rules were like a game of "hide and seek" where the rules said, "You can only hide inside a box that fits a specific grid." If a tiny object (like a distant bird) was smaller than the grid squares, the guard simply couldn't see it. It received zero attention and was ignored during training.
The YOLO26 Fix: They introduced STAL (Small-Target-Aware Label Assignment). Imagine the guard is given a special magnifying glass for tiny things. Even if a tiny object doesn't fit the standard grid, the system forces the guard to pay attention to it by temporarily "stretching" the grid just enough to cover it.
The Result: The model no longer ignores the smallest, hardest-to-see objects.

4. The "Smart Coach" (MuSGD & Progressive Loss)

The Old Problem: Training these models used to be like a marathon where the coach yelled the same instructions for the whole 600-mile race. It took a long time to get good. Also, the coach treated the "Speedster" worker and the "Maximizer" worker exactly the same, even though they had different jobs.
The YOLO26 Fix:

MuSGD (The Smart Coach): They swapped the old training method for a new optimizer (MuSGD) that learns faster, like a coach who knows exactly how to pace the runner to finish in fewer miles (fewer training epochs).
Progressive Loss (The Curriculum): They changed the training schedule. At the start, they focus on the "Maximizer" worker to build a strong foundation. As the training goes on, they gradually shift the focus to the "Speedster" worker, ensuring the final product is perfectly tuned for the fast, no-check inference.

5. Specialized Skills for Different Jobs

Just because the guard is fast at spotting cars doesn't mean they are good at everything. YOLO26 adds specialized tools for other tasks:

Cutting out shapes (Segmentation): They added a multi-scale system that helps the guard understand the background better, making it easier to trace the exact outline of an object.
Tracking people (Pose Estimation): They added a way to guess how "uncertain" the guard is about a joint (like an elbow). If the elbow is hidden, the model admits it's unsure rather than guessing wildly.
Rotated objects (OBB): For things like ships or planes that aren't always straight up and down, they fixed the math so the model doesn't get confused when an object is tilted.

6. The "Open Vocabulary" Upgrade (YOLOE-26)

Finally, they introduced YOLOE-26, which is like giving the guard a universal translator.

Text Prompting: You can tell the guard, "Find me a red fire hydrant," and it will find it, even if it was never explicitly trained on fire hydrants.
Visual Prompting: You can show the guard a picture of a specific toy, and it will find that toy in the video.
No Prompt: It can also just look around and describe what it sees on its own.

The Bottom Line

YOLO26 is a unified family of models that are faster, lighter, and more accurate than their predecessors. They achieve this by removing unnecessary baggage, fixing the rules for spotting tiny things, and using a smarter training coach. Whether you need the absolute fastest speed (the NMS-free path) or the highest possible accuracy (the NMS path), YOLO26 offers a version that sits at the very top of the "speed vs. accuracy" chart.

Technical Summary: Ultralytics YOLO26

1. Problem Statement

Despite the widespread industrial adoption of the YOLO family for real-time vision, current detectors face four specific limitations that hinder optimal accuracy, efficiency, and deployment flexibility:

NMS Dependency and Suboptimal Training: Most CNN-based detectors rely on Non-Maximum Suppression (NMS) at inference. While YOLOv10 introduced a dual-head design to enable NMS-free inference, it applies identical loss weights to both the dense (one-to-many) and inference (one-to-one) branches throughout training. This leaves the critical one-to-one branch under-optimized relative to what targeted supervision could achieve.
DFL Parameter Bloat and Range Limitation: The Distribution Focal Loss (DFL) module, adopted since YOLOv8, expands bounding-box regression from 4 scalars to $4K$ logits (typically $K=16$ ). This inflates model parameters (e.g., a 12% increase in parameters for YOLO11n) and imposes a finite regression range of $(K-1) \times \text{stride}$ , which becomes restrictive for large objects at high resolutions.
Long Training Schedules: Standard SGD-based recipes require approximately 600 epochs to reach competitive COCO accuracy, making rapid iteration costly. While the Muon optimizer has shown efficiency gains in Large Language Model (LLM) training, it has not been adapted for object detection.
Zero-Assignment for Small Objects: The Task-Aligned Learning (TAL) label assignment strategy relies on geometric containment within ground-truth boxes. For tiny objects, feature-map downsampling can result in zero valid anchor centers inside the box, leading to zero positive assignments and a complete lack of gradient signal for these targets.

2. Methodology

YOLO26 addresses these limitations through a coordinated architecture and training pipeline, built upon the YOLO11 foundation.

2.1. Architecture: Dual-Head NMS-Free Design

YOLO26 employs a dual-head design to support both end-to-end inference and conventional dense prediction:

One-to-One Head (Default): Produces a fixed-size set of predictions (up to 300 per image) without NMS. It uses a strict one-to-one label assignment (TAL with $topk=7$ followed by $topk2=1$) to learn a single matched prediction per ground-truth instance.
One-to-Many Head: Retains standard dense prediction with richer supervision ($topk=10$) for scenarios where maximum accuracy is prioritized over post-processing overhead.
DFL Removal: The Distribution Focal Loss module is entirely removed. Regression is performed via direct 4-scalar output with an unconstrained range, significantly reducing head parameters and FLOPs.

2.2. Training Pipeline

Three complementary techniques are jointly applied to improve accuracy and convergence:

MuSGD Optimizer: A hybrid optimizer combining Muon (from LLM training) with standard SGD-momentum. Muon applies momentum updates followed by lightweight orthogonalization (via Newton–Schulz iterations) to improve update conditioning. MuSGD applies this to multi-dimensional parameters (convolutions, linear weights) while using pure SGD for 1D parameters (biases, scales).
Progressive Loss: A curriculum-style reweighting strategy that addresses the optimization asymmetry between the two branches. The loss weight for the dense (one-to-many) branch is linearly decayed from 0.8 to 0.1, while the one-to-one branch weight increases from 0.2 to 0.9. This shifts supervision emphasis toward the inference-time head as training progresses.
STAL (Small-Target-Aware Label Assignment): To prevent zero-positive assignments for tiny objects, STAL decouples candidate selection geometry from regression geometry. During candidate filtering, the ground-truth box dimensions are clamped to a reference size ( $s_{ref}$ , typically the next pyramid stride) if they fall below the minimum stride. This ensures anchor coverage for small targets while preserving the original box geometry for final regression and scoring.

2.3. Task-Specific Extensions

Beyond detection, YOLO26 introduces specialized designs for other tasks:

Instance Segmentation: Uses a multi-scale prototype pathway (fusing features from multiple pyramid levels) and an auxiliary semantic segmentation loss (training-only) to improve mask quality without inference overhead.
Pose Estimation: Replaces standard coordinate regression with Residual Log-Likelihood Estimation (RLE). This models per-joint uncertainty using a normalizing flow, down-weighting ambiguous or occluded keypoints.
Oriented Bounding Box (OBB): Adopts a "long-edge" angle definition ( $[-45^\circ, 135^\circ)$ ) to resolve boundary discontinuities. It introduces an aspect-ratio-aware angle loss specifically for square objects to handle angle ambiguity.
Open-Vocabulary (YOLOE-26): Extends the family to open-vocabulary scenarios using a stronger backbone, an upgraded text encoder (MobileCLIP2), a pseudo-label data engine to refine annotations, and decoupled segmentation training.

3. Key Results

YOLO26 spans five scales (n/s/m/l/x) and achieves state-of-the-art accuracy-latency trade-offs on the COCO val2017 benchmark.

Detection Performance: YOLO26 achieves 40.9–57.5 mAP on COCO with T4 TensorRT FP16 latencies ranging from 1.7 ms (n) to 11.8 ms (x). It advances the Pareto front over prior YOLO versions (YOLO11, YOLOv10, etc.) and other real-time detectors.
- Compared to YOLO11, YOLO26 improves COCO AP by 1.6–2.8 points across scales.
- Removing DFL alone reduced parameters by 12% and FLOPs by 20% for the nano model, with accuracy recovered and improved via the new training pipeline.
Small Object Detection: STAL significantly improves small-object detection (APS), recovering detections missed by standard TAL.
Task-Specific Gains:
- Segmentation: Up to +3.7 mask AP improvement over YOLO11.
- Pose Estimation: Up to +7.2 pose AP improvement over YOLO11.
- OBB Detection: Up to +3.4 mAP improvement on DOTA-v1.0.
Open-Vocabulary (YOLOE-26): The YOLOE-26x model reaches 40.6 AP on LVIS minival under text prompting, surpassing DetCLIP-T by +6.2 AP.
Efficiency: The architecture achieves up to 43% faster CPU inference compared to previous versions (YOLO26n vs. YOLO11n) and supports export to 19 non-PyTorch targets (TensorRT, ONNX, CoreML, etc.).

4. Significance and Claims

The paper claims that YOLO26 represents a unified step forward in real-time vision by directly addressing the structural and training inefficiencies of the YOLO lineage. Its primary significance lies in:

NMS-Free End-to-End Viability: Demonstrating that a dual-head architecture with Progressive Loss can achieve NMS-free inference accuracy that rivals NMS-based methods, simplifying deployment pipelines.
Efficiency through Simplification: Proving that removing the complex DFL module does not degrade performance but rather enhances it when paired with unconstrained regression and improved training strategies, resulting in lighter models suitable for edge devices.
Optimization and Labeling Synergy: Establishing that combining advanced optimizers (MuSGD) with curriculum-based loss weighting (Progressive Loss) and robust label assignment (STAL) creates a synergistic effect that accelerates convergence and improves small-object detection without increasing model size.
Unified Multi-Task Framework: Providing a single pipeline that supports detection, segmentation, pose, classification, and OBB detection with consistent gains across all tasks, alongside a robust open-vocabulary extension.

The authors emphasize that YOLO26 is designed not only for strong optimization behavior but for practical inference integration across heterogeneous hardware, maintaining the deployment universality that has made the YOLO family the industry standard.