🤖 AI

OpenURMA: A Clean-Room Open Implementation of the Unified Bus Protocol

This paper introduces OpenURMA, the first clean-room open-source implementation of Huawei's Unified Bus protocol, which demonstrates through RTL, SystemC, and gem5 simulations that decoupling per-application state from transport state reduces 64-byte remote fetch latency by 4.37x and increases throughput by 2.80x compared to traditional RoCEv2 baselines.

Original authors: Bojie Li

Published 2026-05-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Bojie Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Peripheral" Bottleneck

Imagine a modern datacenter as a massive office building where thousands of employees (applications) need to talk to each other instantly.

Currently, the standard way they talk is like this:

The Peripheral Problem: The network card (NIC) is treated like a peripheral device, similar to a printer or a mouse plugged into a computer. It sits "outside" the main brain (the CPU).
The Queue System: To send a message, an employee has to write a note (a "Work Request"), walk it to the network card's mailbox (the "Doorbell"), and wait. The network card then picks it up, processes it, and writes a "Done" note back to the mailbox.
The Traffic Jam: Because the network card is "outside," every single note has to cross a busy highway (the PCIe bus) four times just to get sent and confirmed.
The Memory Crisis: The network card has to keep a personal file folder for every single pair of employees talking to each other. If you have 1,000 employees talking to 1,000 others, the card needs 1 million folders. It runs out of desk space (on-chip memory) and has to start running to the filing cabinet in the basement (host RAM) to grab files. This slows everything down to a crawl.

The Result: Even though the internet cables (the "wire") are incredibly fast, the whole system is stuck in traffic because of how the network card is connected and organized.

The Solution: Huawei's Unified Bus (UB)

Huawei proposed a new design called Unified Bus (UB) (in their Ascend 950 chip) to fix this. They didn't just patch the old system; they changed the rules of the road.

1. Moving the Office Inside the Building
Instead of the network card being a peripheral outside the building, UB puts the network controller inside the building on the main hallway (the on-chip bus).

Analogy: Instead of walking out to the lobby to mail a letter, you can now just hand it to a colleague standing right next to your desk. You don't need to cross the highway four times; you just take one step.

2. Splitting the Filing System
The old system kept one giant folder for every pair of people talking. UB splits this up:

Jetty: A small card for your side of the conversation.
TP Channel: A shared card for the other person's side.
Analogy: Instead of needing a unique folder for every pair of people (which explodes in size), you just need a card for yourself and a shared card for the person you are talking to. The total number of folders grows slowly (additively) instead of exploding (multiplicatively). This keeps the network card's desk space from overflowing.

3. The "Load/Store" Shortcut
Because the controller is now inside the building, the CPU can talk to it directly using standard instructions (like LOAD or STORE), just like reading a variable in a program.

Analogy: You don't need to fill out a form, walk to the mailbox, and wait for a receipt. You just grab the data you need instantly.

4. Optional Rules (Opt-in Ordering)
The old system forced strict rules: "You must receive messages in the exact order they were sent, no matter what." This is slow and unnecessary for many tasks.

Analogy: UB says, "We can deliver messages as fast as possible. If you really need them in order, just raise your hand and we'll slow down to sort them. If you don't care, we'll just dump them in your inbox as they arrive." This saves time for everyone who doesn't need strict order.

What OpenURMA Did

Huawei's chip exists, but it's a "black box." No one can see how it works inside, measure it, or build on it.

OpenURMA is the first open-source, clean-room implementation of this new system.

"Clean-room" means they built it from scratch using only the public rulebook (the specification), without looking at Huawei's secret code.
They built three versions to test it:
1. RTL: The actual blueprint for hardware chips (tested on an FPGA board).
2. SystemC: A super-detailed computer simulation of the network.
3. gem5: A full computer simulation running a real operating system.

They compared their new system (OpenURMA) against the old standard (RoCEv2) using the exact same tools to ensure a fair fight.

The Results: How Much Faster?

The paper claims massive improvements, specifically for small, fast operations (like fetching a tiny piece of data):

Speed: The new system is 4.37 times faster (about 500 nanoseconds) than the old system (about 2,186 nanoseconds) for a standard data fetch.
Throughput: It can handle 2.8 times more data per second.
Efficiency: It uses very little space on the chip (only about 14% of the available space on a test board).
Scalability: As you add more users (from 1 to 1,024), the old system slows down drastically because it runs out of desk space. The new system stays fast and steady because its filing system is efficient.

Why This Matters (According to the Paper)

The paper argues that the current way we connect computers in datacenters (treating the network card as a peripheral) is fundamentally broken for modern AI and big data workloads.

The "Coherent" Alternative: There are other technologies (like CXL or NVLink) that try to make memory look "shared" and "coherent." However, the paper argues these technologies break down when you try to scale them up to large clusters (many racks of servers) because they require too much state tracking and cannot handle network errors well.
The UB Advantage: UB accepts that the network isn't perfect and doesn't try to force "perfect coherence" across the whole building. Instead, it gives the application the tools to manage order only when needed, allowing it to scale to thousands of servers without the performance collapse seen in other systems.

Summary

Think of the old system as a post office where you have to walk to the counter, fill out a form, wait in line, and get a receipt for every single letter.
Think of the new OpenURMA system as a private messenger who sits right next to you, knows exactly who you are talking to, and can drop a letter in your hand instantly without any paperwork, unless you specifically ask for a receipt.

The paper proves that this new way of organizing the network is not just a theory, but a working, open design that is significantly faster and more scalable than what we use today.

Technical Summary: OpenURMA

Problem Statement
Modern datacenter RDMA (Remote Direct Memory Access) is structurally bottlenecked by the "peripheral-NIC" abstraction inherited from InfiniBand and RoCE. In this model, the Network Interface Card (NIC) resides behind the PCIe bus, and communication follows a Queue-Pair (QP) programming model. This design imposes two critical costs that have become untenable for AI-training workloads characterized by massive fanouts (thousands of GPUs) and small message sizes (64-byte gradients):

State Explosion: A QP fuses application identity with transport reliability, requiring per-connection state for every (local application, remote endpoint) pair. At a 1024x1024 scale, this requires ~537 MB of state, exceeding on-chip SRAM and forcing operations to pay a PCIe round trip to refetch context.
PCIe Traversal Latency: Because the NIC is a peripheral, every operation crosses the PCIe boundary four times (doorbell MMIO, WQE DMA fetch, CQE DMA write, and CPU poll-miss). On a 64-byte operation, these traversals consume ~1.5 µs of a ~2 µs round trip, inflating latency an order of magnitude beyond the wire speed.

Prior work (software patches like FaSST, hardware optimizations like SRNIC, or programmable substrates) has addressed symptoms but left the fundamental abstraction intact.

Methodology
The paper presents OpenURMA, the first clean-room open implementation of Huawei's Unified Bus (UB) specification (2025), which replaces the peripheral-NIC abstraction. UB decouples per-application endpoint state from per-host transport state, moves the controller onto the on-chip bus, and exposes ordering as an opt-in feature.

The implementation is realized using the OpenClickNP toolchain, which lowers a single source description into three modeling tiers for rigorous, apples-to-apples comparison against a matched OpenRoCE (RoCEv2 RC) baseline:

Synthesisable RTL: Implemented on an Alveo U50 FPGA, closing at 322 MHz.
Cycle-Level Simulation: A two-node SystemC simulator modeling both endpoints, wire, and memory hierarchies.
Full-System Scaffold: A gem5 environment running real ARM binaries and a Linux driver against the SystemC NIC.

The evaluation methodology explicitly accounts for bidirectional costs, including target-side NIC-to-DRAM transfers and PCIe traversals, avoiding the "submit-only" latency measurements common in prior literature.

Key Architectural Moves
OpenURMA implements four architectural moves defined in the UB spec:

Layer Split (Bounded State): Separates the transaction layer (per-application Jetty) from the transport layer (per-host TP Channel). State scales additively as $O(N+M)$ rather than multiplicatively as $O(N \cdot M)$ .
On-Chip Controller: With state bounded to ~110 KB (fitting in on-chip SRAM), the controller resides on the CPU's on-chip bus, eliminating the PCIe boundary.
Load/Store Path: The CPU can access remote memory via native ld/st instructions directly to the controller, bypassing work queues, doorbells, and DMA.
Opt-in Ordering: Ordering is exposed as four orthogonal axes (service mode, execution order, fence, completion order) that applications opt into. Gating logic reuses existing counters, adding zero pipeline cycles for unordered operations.

Key Results
On the canonical 64-byte remote fetch (LOAD on UB vs. READ on RoCEv2):

Latency: UB's load/store path achieves ~500 ns end-to-end latency, a 4.37× reduction compared to the matched RoCE DMA baseline (2186 ns). Even on verbs requiring the work-queue path, UB is ~2.2× faster due to the elimination of PCIe traversals.
Throughput: UB sustains 2.80× higher work-request throughput than OpenRoCE.
State Scaling: At 1024x1024 endpoints, OpenURMA uses 110.6 KB of state versus 536.9 MB for RoCE, a 4,855× reduction. This prevents the "latency cliff" where RoCE performance degrades sharply once connection counts exceed ~23 (SRAM spill point).
Ordering Cost: Opt-in ordering is "near-free"; operations bypassing gating incur zero pipeline cycles.
Reliability & Congestion: OpenURMA implements selective acknowledgement (SACK) and a C-AQM congestion controller, showing superior goodput and tail latency under loss compared to Go-Back-N and DCQCN.
Scale-Out vs. Coherent Fabrics: The paper argues that coherent fabrics (CXL, NVLink) cannot scale to rack-level due to directory state growth ( $O(N)$ ) and invalidation broadcast costs. UB's non-coherent, opt-in approach avoids these cliffs, maintaining flat latency as cluster size increases.

Significance and Claims
The paper claims that OpenURMA provides the first empirical validation of the Unified Bus protocol's architectural promises without relying on closed silicon. The significance lies in demonstrating that the three architectural commitments—bounded state, on-bus controller, and opt-in ordering—are mutually load-bearing:

Bounded state is required to keep the controller on-chip.
The on-chip controller is required to enable the load/store path.
The layer split is required to make opt-in ordering cost-free.

The authors conclude that OpenURMA replaces the peripheral-NIC abstraction with a peer-to-peer model that scales to AI-training workloads, offering a path to sub-microsecond remote memory access that is structurally impossible under the current PCIe/QP paradigm. The implementation is released as open-source RTL and simulation harnesses to enable further research and prototyping.