⚡ electrical engineering

CUDA MPC: A GPU-Native Solver for Model Predictive Control

This paper introduces CUDA MPC, a GPU-native framework that co-designs a parallel-in-horizon ADMM algorithm with fused CUDA kernels and shared-memory optimization to achieve real-time Model Predictive Control on complex, high-dimensional systems with significantly lower latency and higher scalability than existing CPU and tensor-framework solvers.

Original authors: Babak Akbari, Melissa Greeff

Published 2026-08-05

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Babak Akbari, Melissa Greeff

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to do something tricky, like parking a car in a tight spot or guiding a swarm of drones through a forest without them crashing into each other. To do this well, the robot needs a brain that can look ahead, predict what will happen next, and constantly adjust its plan to avoid obstacles. This is called Model Predictive Control (MPC). Think of it like a chess player who doesn't just make one move, but simulates a hundred possible future games in their head before deciding on the best move. The problem is, doing this simulation takes a lot of brainpower. If the robot is moving fast or the world is complicated, the math gets so heavy that a standard computer brain (a CPU) can't finish the calculations before the robot needs to make its next move. It's like trying to solve a giant puzzle while running a marathon; by the time you finish the puzzle, you've already tripped.

For a long time, scientists tried to speed this up by using Graphics Processing Units (GPUs), the super-fast chips usually found in video game computers. However, most people just used GPUs like a fancy calculator, asking them to do small math tasks one after another. This is inefficient because the robot has to constantly send data back and forth between the main computer and the GPU, which creates a traffic jam. The new paper introduces a smarter way to use these powerful chips, not just as a calculator, but as a dedicated, high-speed factory that keeps all the work inside its own walls.

The paper presents CUDAMPC, a new system designed from the ground up to run Model Predictive Control directly on a GPU. Instead of treating the GPU as a simple helper that does math in pieces, the authors built a "fused" engine that keeps the entire planning process inside the chip's fastest memory. To understand how this works, imagine a long line of people passing a bucket of water down a chain to put out a fire. In the old way, every person would have to shout to the person next to them, wait for a reply, and then pass the bucket, creating a lot of noise and delay. In the CUDAMPC approach, the line is split into small groups. Each group works together silently and quickly, passing the bucket only at the very ends of the group. This allows the whole chain to move much faster because everyone is working in parallel without waiting for the whole line to stop and talk.

The researchers tested this new system against the best existing computer programs (like acados and CasADi) and other GPU methods. They found that CUDAMPC is incredibly fast, especially for long planning horizons. In one test, it solved a complex parking problem with a "lookahead" of 100 seconds in just 0.1 seconds, a task that took other solvers several seconds or failed completely. In a test involving a swarm of 10 agents trying to coordinate without crashing, the old CPU solvers took over 3.5 seconds per step (too slow for real-time control) or couldn't find a solution at all, while CUDAMPC did it in milliseconds. The paper shows that by keeping all the intermediate data inside the chip's fast memory and synchronizing only the necessary neighbors, the system can solve problems that were previously impossible to do in real-time. The authors suggest that this approach allows robots to plan much further ahead, leading to safer and more agile behavior in complex environments.

Technical Summary: CUDAMPC

Problem Statement

Model Predictive Control (MPC) is a powerful methodology for enforcing state and input constraints while optimizing multivariable performance. However, its reliance on online optimization creates a significant computational burden, particularly for fast-sampled, high-dimensional, or highly nonlinear robotic systems. Traditional CPU-based solvers (e.g., acados, CasADi) often cannot compute accurate control actions within tight sampling intervals, forcing compromises in horizon length, model fidelity, or optimization accuracy.

While GPUs offer thousands of parallel cores, existing GPU-based MPC implementations typically treat the device merely as a linear-algebra accelerator. Approaches using tensor frameworks (e.g., PyTorch, JAX) express iterative optimization as sequences of framework-level operations. This results in repeated kernel launches and high-latency global memory transfers, which dominate solve times at high frequencies and erode the parallel advantages of the GPU. Furthermore, many existing GPU methods parallelize only specific numerical kernels (e.g., Newton-system solves) or rely on sampling-based approaches (e.g., MPPI) that lack feasibility certificates and struggle with high-dimensional constraints.

Methodology

The paper introduces CUDAMPC, a GPU-native MPC framework co-designed at the algorithmic, execution, and memory levels to overcome these limitations. The core methodology involves three tightly coupled design layers:

Algorithmic Level (Parallel-in-Horizon ADMM):
The framework utilizes a parallel-in-horizon Alternating Direction Method of Multipliers (ADMM) splitting. The nonlinear MPC problem is first addressed via Sequential Convex Programming (SCP), linearizing dynamics and constraints. The resulting convex subproblem is then split using ADMM consensus variables. This formulation decouples the optimization across the prediction horizon, allowing independent stagewise updates that are coupled only through nearest-neighbor consensus.
Execution Level (Fused CUDA Kernel):
Unlike tensor-framework implementations that launch separate kernels for each ADMM operation, CUDAMPC fuses the entire iterative solve into a single, custom CUDA kernel.
- No Host Intervention: The complete inner ADMM loop executes natively on the GPU without returning control to the host between iterations.
- Neighbor-Local Synchronization: Instead of using expensive grid-wide barriers (grid.sync()) that scale with the number of blocks, CUDAMPC employs a localized atomic-flag protocol. Blocks synchronize only with their immediate spatial neighbors (forward and backward propagation of boundary variables). This reduces coordination costs to be independent of the total number of horizon blocks and introduces a bounded delay (partially asynchronous ADMM), which is proven to converge.
Memory Level (Block-Resident Storage):
Following the I/O-aware principles of FlashAttention, the framework minimizes transfers between global and on-chip memory.
- Shared Memory Residency: Optimization variables (primal, dual, and consensus) and intermediate computations are loaded into low-latency, on-chip shared memory at the start of the kernel and remain resident throughout the solve.
- Resource Allocation: The horizon is partitioned into sub-horizons assigned to CUDA thread blocks. The sub-horizon length is dynamically calculated based on the device's thread and shared-memory limits, ensuring the working set fits entirely on-chip. Global memory is used only for initial loading, final output, and boundary variable exchange.

Key Contributions

The paper claims three primary contributions:

CUDAMPC Framework: A GPU-native architecture that executes parallel-in-horizon ADMM within a single, shared-memory-resident CUDA kernel.
Neighbor-Local Synchronization Protocol: A mechanism replacing grid-wide barriers with pairwise atomic flags, ensuring coordination costs remain independent of the number of horizon blocks.
Benchmark Study: A comprehensive evaluation against state-of-the-art CPU and GPU solvers on six nonlinear robotics problems, demonstrating that the increased real-time-feasible horizon enables closed-loop capabilities (e.g., collision avoidance, swarm coordination) that CPU baselines cannot achieve at any horizon they can run.

Results

The paper evaluates CUDAMPC against CPU solvers (CasADi with Ipopt, acados with HPIPM) and other GPU methods (Primal-Dual iLQR, GPU-SLS, and tensor-framework implementations of $\pi$ MPC) across six benchmarks: Pendulum, Cart-Pole, Car Parking, Truck-Trailer, Quadcopter-Pole, and Centralized Swarm.

Performance vs. Tensor Frameworks: Against PyTorch and Julia implementations of $\pi$ MPC on a linearized aircraft model, CUDAMPC achieved speedups ranging from 136 $\times$ to 965 $\times$ . This was attributed to reducing kernel launches from ~172,000 to 1 per solve and reducing VRAM traffic from ~17 GB to ~31 MB.
Scalability: On nonlinear benchmarks, CUDAMPC's solve-time scaling with horizon length ( $N$ ) was significantly superior to CPU methods. For example, on the Pendulum problem at $N=1000$ , CUDAMPC was 20.1 $\times$ faster than the next-fastest solver.
Real-Time Feasibility: CUDAMPC sustained real-time rates at horizons one to two orders of magnitude longer than CPU solvers.
- Car Parking: Solved an optimization-based collision-avoidance problem with a 100-second lookahead within a 0.1-second sampling interval.
- Centralized Swarm (10 agents): CUDAMPC was the only solver evaluated that achieved both real-time execution (~28 ms) and collision-free coordination. CPU solvers (acados, CasADi) failed to find feasible solutions within the sampling interval (taking 3.5s and 4.5s respectively), and the GPU-based Primal-Dual iLQR, while real-time, resulted in deadlocked local minima due to the difficulty of tuning soft penalty constraints for hard collision avoidance.

Significance

The paper posits that CUDAMPC represents a shift from using GPUs merely as accelerators for linear algebra to a holistic co-design of the optimization algorithm and hardware architecture. By minimizing host intervention and global memory traffic, the framework unlocks the ability to run long-horizon, constraint-heavy MPC problems in real-time. The significance lies not just in raw speed, but in enabling control strategies (such as centralized multi-agent coordination and long-horizon collision avoidance) that were previously computationally intractable for real-time systems, thereby expanding the operational envelope of advanced robotic platforms.