End2Race: An End-to-End Learning Framework for Multi-Vehicle Autonomous Racing
This paper introduces End2Race, an end-to-end learning framework for multi-vehicle autonomous racing that achieves sub-millisecond inference latency and outperforms existing rule-based and single-vehicle learning methods by demonstrating robust generalization, high speeds, and adaptive overtaking in head-to-head competitions.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The world of self-driving cars is often imagined as a quiet journey down a straight highway, where the vehicle simply needs to recognize a stop sign or a pedestrian. But the true test of an autonomous system's intelligence lies in the chaos of a race track. Here, vehicles move at high speeds, constantly reacting to one another in a dynamic environment where a split-second delay can mean the difference between a clean pass and a crash. To study this high-stakes environment without the risk of destroying expensive real-world cars, researchers use a scaled-down platform known as F1Tenth. These are one-tenth the size of a real car, equipped with sensors that scan their surroundings like a radar, allowing them to navigate tracks at speeds approaching 18 meters per second. The challenge is not just to drive fast, but to drive fast while making complex decisions about when to speed up, when to slow down, and how to squeeze past a rival vehicle without touching it.
For years, the most successful strategies in these racing competitions have relied on rigid, pre-programmed rules. Engineers build a digital map of the track before the race begins, calculating the perfect line for the car to follow. While this works well on a known track, it fails the moment the car encounters a new layout or a rival that does not follow the expected path. Other methods try to react instantly to what the car sees right now, but they often lack the foresight to plan a smooth maneuver, leading to jerky, unstable driving. A third approach uses machine learning, where a computer learns by watching experts or by trial and error. However, most of these learning systems have been tested only on single cars driving alone, or they have been too slow to react in real-time, forcing them to drive at speeds far too low to be competitive. The gap remained: no one had successfully taught a computer to learn how to race head-to-head against another car at high speeds while reacting instantly.
To bridge this gap, a team of researchers introduced a new system called End2Race. This framework is designed to teach an autonomous car how to overtake another vehicle in real-time, using a learning process that mimics how a human driver might learn: by first watching an expert and then practicing on its own. The researchers set up a virtual training ground where a car, known as the ego vehicle, is paired with an opponent. They created 720 distinct scenarios where the opponent drives along different paths and at different speeds, forcing the learning car to figure out how to find a gap, move into it, and pass safely. The system does not rely on a pre-drawn map of the track; instead, it learns to drive solely based on the raw data from its sensors and the speed of its own engine.
The core of this system is a neural network, a type of computer program designed to recognize patterns, which was built to be incredibly fast. In the world of autonomous racing, time is measured in milliseconds, and a delay of even a fraction of a second can cause a car to miss a turn or collide. The researchers engineered their network to make a decision in less than one millisecond, a speed fast enough to keep up with the rapid changes of a race. To train this network, they first used a method called behavioral cloning, where the computer analyzed 720 eight-second expert demonstrations to learn the basics of steering and speed. However, simply copying the expert was not enough, because the expert sometimes made mistakes or failed to adapt to the unpredictable nature of a moving opponent. To fix this, the team then let the computer race against itself across these scenarios 500 times, using a technique called reinforcement learning. In this phase, the car was rewarded for moving forward quickly and penalized for crashing, allowing it to refine its own strategies and learn how to handle situations the expert had never encountered.
The results of this training were striking. When tested on a track it had never seen before, the learned car could drive at an average speed of over 8.5 meters per second, significantly faster than previous learning-based methods which were often limited to speeds below 3 meters per second. More importantly, when placed in a head-to-head race against an opponent, the new system succeeded in overtaking the rival in nearly 97 percent of attempts while maintaining a safety record of nearly 98 percent. This performance was far superior to older rule-based systems and other learning methods, which either crashed frequently or failed to pass the opponent at all. The system proved capable of adapting to different opponents, even when those opponents drove in ways the computer had never seen before, such as using a different algorithm to avoid obstacles.
The researchers also examined exactly how the car made its decisions. In one scenario, the car would spot a slower opponent ahead, smoothly shift to the side to align with a gap, and then accelerate to pass. In another, it would brake slightly before a sharp turn to maintain control, then accelerate hard as soon as it exited the curve, a maneuver that mirrors the techniques used by professional human racers. The system was not perfect; in rare cases where the track suddenly narrowed while the two cars were side-by-side, the car would collide because it could not predict the future shape of the track. However, these failures were far less frequent than with other methods, and the overall performance demonstrated that the car had learned a robust, adaptable style of driving.
This work suggests that the future of autonomous racing, and potentially high-speed autonomous driving in general, lies in systems that can learn from experience rather than relying on static rules. By combining a fast, efficient neural network with a two-stage training process, the researchers showed that a machine can learn to navigate complex, dynamic interactions with a level of speed and safety that was previously unattainable. The system does not need a map to know where to go; it simply needs to see the world, understand the flow of traffic, and react in the blink of an eye. As the researchers prepare to test this system in real-world competitions against other teams, the implications extend beyond the race track, offering a glimpse into how autonomous vehicles might one day handle the unpredictable chaos of shared roads with confidence and skill.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.