💰 quantitative finance

Packets, Transactions and Queues: Design Principles for HFT Systems from a Measurement Study of CME Market Data

By analyzing over a year of CME market data, this paper challenges the conventional single-threaded HFT design by demonstrating that while one thread suffices for sub-period packet processing, a two-stage threaded architecture can significantly reduce queuing tails caused by transaction bursts, provided the split shortens the system's slowest stage.

Original authors: Vincent Maciejewski

Published 2026-09-29✓ Author reviewed ⓘ

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Vincent Maciejewski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of high-frequency trading, where computers buy and sell stocks in fractions of a second, speed is not just an advantage; it is the entire game. These systems operate on a simple premise: if you can process market information faster than anyone else, you can profit from tiny price differences before they disappear. To do this, engineers build specialized software that listens to a constant stream of data from stock exchanges, decodes it, and makes decisions in microseconds. For years, the industry has operated under a strict rule: keep the most critical part of this software on a single processor core. The logic was that moving data between different cores, or "threads," was too slow and risky, adding delays that would ruin the system's speed. This approach treated the software like a single, focused worker who never hands off a task, believing that any interruption would cost more than the work itself.

However, this long-held belief relied on an assumption about how market data arrives: that it comes in a steady, random stream, like raindrops falling at unpredictable intervals. If that were true, the single-worker approach would indeed be the fastest. But what if the data doesn't fall randomly? What if it arrives in sudden, intense bursts, where thousands of updates hit the system in the blink of an eye? A new measurement study of real market data from the Chicago Mercantile Exchange suggests that the old rule might be wrong for the busiest moments. By tracking billions of data packets over more than a year, researchers discovered that market data does not arrive randomly. Instead, it arrives in tight clusters, where one event triggers a rapid succession of others, creating a "self-exciting" pattern. This discovery changes the math of speed. It turns out that when data arrives in these specific, clustered bursts, splitting the work across multiple processors can actually make the system faster and more reliable, provided the system is designed to handle the rhythm of the bursts correctly.

The researchers began by looking at the raw data stream as it travels from the exchange's matching engine, where orders are processed, to the traders' computers. They followed every single packet of data, noting exactly when it left the exchange and when it arrived. They found that the exchange's system acts like a gatekeeper with a fixed speed limit. Even when the matching engine processes orders incredibly fast—sometimes within a fraction of a microsecond of each other—the exchange's data publisher cannot send them all at once. It sends them one by one, with a minimum gap of about 7.5 microseconds between each packet. This creates a train of data packets that arrive at the trader's computer with a steady, rhythmic spacing, regardless of how chaotic the activity was at the source.

This rhythmic arrival is the key to the new findings. The researchers built a computer simulation to test how different software designs would handle this specific rhythm. They compared the traditional single-threaded approach, where one processor does all the work, against a multi-stage pipeline, where the work is split among several processors working in sequence. In their simulation, they fed the system the exact timing of the real data packets. The results were clear: for tasks that take longer than the 7.5-microsecond gap between packets, the single-threaded approach creates a massive backlog. When a burst of data arrives, the single processor gets overwhelmed, and the delay for the last few packets in the burst grows to be dozens of times longer than the task itself. This delay is the "tail" that traders fear, as it means their decisions are made too late.

In contrast, the multi-stage pipeline handled these bursts with ease. By splitting the work, the system could process the incoming train of packets in parallel. While the first processor was decoding the first packet, the second was already working on the second, and so on. This allowed the system to drain the backlog much faster, keeping the delay for every packet low and consistent. The simulation showed that for tasks taking 16 microseconds or longer, splitting the work reduced the worst-case delays by a factor of ten or more, with only a tiny penalty for the typical, non-bursty moments. The researchers confirmed that this improvement was not due to the sheer volume of data, but specifically to the clustered, bursty nature of the arrival times. When they simulated the same amount of data arriving randomly, the multi-stage system offered no advantage, and the single-threaded system remained efficient.

The study also ruled out several other potential causes for the delays. They found that the size of the data packets or the number of messages inside them was not the primary driver of the slowdown. Even when they rearranged the data to remove the bursts but kept the same number of packets, the massive delays disappeared. This proved that the problem was purely about the timing of the arrivals. The researchers also looked at the exchange itself to understand why the data arrived in these clusters. They found that the exchange's matching engine often processes multiple orders almost simultaneously, likely because many traders are reacting to the same market event at the same time. However, the exchange's publisher then spaces these out, creating the rhythmic train that the traders' systems must handle.

For the designers of these trading systems, the paper offers a clear, data-driven guide. If a system's processing time is shorter than the 7.5-microsecond gap between packets, the old rule still applies: keep it on a single thread. There is no benefit to splitting the work, and it only adds unnecessary complexity. But if the processing time is longer than that gap, the single-threaded approach will fail during bursts, and the system should be split into multiple stages. The researchers emphasize that the goal is not to use as many processors as possible, but to ensure that the slowest part of the process is fast enough to keep up with the exchange's rhythm. They also found that the specific arrangement of the processors matters less than ensuring that the slowest stage is handled efficiently.

This work does not claim to have solved every problem in high-speed trading, nor does it suggest that the single-threaded approach is obsolete. It simply provides a precise measurement of when that approach stops working and when a different design becomes necessary. By measuring the real world rather than relying on theoretical models, the researchers have given engineers a concrete threshold to measure against. They have shown that the nature of the data stream—specifically its tendency to arrive in self-exciting bursts—dictates the best way to build the software that consumes it. The lesson is that in the high-speed world of finance, understanding the rhythm of the data is just as important as the speed of the computer.

Technical Summary: Packets, Transactions and Queues

Problem Statement

High-frequency trading (HFT) system design faces a fundamental architectural choice: should the critical "hot path" (ingress to order book application) be implemented as a single-threaded event loop or as a multi-stage pipeline (tandem) of actors running on separate cores?

The prevailing industry consensus favors the single-threaded approach, arguing that the cost of inter-thread communication (mailbox hops, cache-line transfers, and context switches) adds a fixed latency penalty that outweighs any throughput benefits. However, this paper challenges that consensus by arguing that the optimal architecture depends on the statistical properties of the incoming market data feed. Specifically, it investigates whether the self-exciting, clustered nature of real exchange feeds creates a queueing tail that a multi-stage pipeline can mitigate, and if so, under what conditions.

Methodology

The study employs a rigorous measurement-driven approach combining empirical data analysis, analytical theory, and large-scale simulation.

Data Corpus: The author analyzed over one year of CME MDP3 market data for the NQ front-month contract (approx. 3 billion packets). The dataset includes nanosecond-resolution timestamps from three distinct stages: the matching engine (transactTime), the market-data publisher (sendingTime), and the receiver.
Measurement of the Feed: The author characterized the packet arrival process using point-process diagnostics (Fano factor, Hurst exponent) and Hawkes process fitting. They traced the relationship between matching-engine transactions and the resulting packets to determine if clustering originates from the exchange's packetization or the market's transaction timing.
Analytical Framework: The paper derives two closed-form results for deterministic-service tandem queues:
- Burst-Limit Throughput: An $N$ -stage tandem clears an isolated burst $N$ times faster than a single server.
- Pathwise Reduction: The end-to-end wait of an $N$ -stage tandem is bounded by the wait of a single server with the slowest stage's service time, plus the hop costs.
Simulation: A Lindley recursion simulator was built to replay the real packet arrival stream through $N$ -stage pipelines ( $N \in \{1, 2, 4, 8\}$ ) with varying service times ( $T \in \{2, \dots, 128\} \mu s$ ).
Null Streams: To isolate causal factors, the author constructed "null" streams that removed specific ingredients of the real data (e.g., shuffling gap order, randomizing transaction times, merging transactions into single packets) to determine which properties drive queueing tails.
Cross-Validation: Simulation results were cross-validated against a live production receiver on other instruments to ensure the findings hold in a real-world environment.

Key Contributions

1. Characterization of the Exchange Feed

Publisher as a Queue: The exchange's market-data publisher behaves as a deterministic queue with a service period of approximately 7.5 µs. It sends at most one packet per period.
Transaction Clustering: The clustering observed in the packet stream is inherited from the matching engine's transactions, not from the exchange's packetization. Almost every transaction fits in a single packet.
Engine vs. Publisher Clocks: The matching engine processes consecutive transactions within fractions of a microsecond (often < 1 µs). However, the publisher spaces these events out to its 7.5 µs period. Thus, the receiver sees a "train" of packets spaced by the publisher period, not a burst of simultaneous packets.
Backlog Handling: When the engine bursts, the publisher's delay increases (up to milliseconds), but it drains the backlog by sending more packets at the fixed period, not by packing more messages into larger packets.

2. Theoretical Results

The Reversal of the Single-Thread Rule: The paper establishes that the choice between single-thread and multi-thread architectures is not absolute but depends on the service time relative to the publisher period ( $s_{pub}$ $s_{p u b}$ ).
- Below $s_{pub}$ : If the service time per packet is less than the publisher period, the receiver never queues on packet arrivals, regardless of market bursts. In this regime, splitting adds hop costs with no benefit.
- Above $s_{pub}$ : If the service time exceeds the publisher period, the receiver queues on every packet of a burst. Here, the self-exciting nature of the feed creates a significant tail.
Tail Compression: For service times above $s_{pub}$ , splitting the task into $N$ stages reduces the queueing tail significantly. The tail excess scales roughly with $N^{-\gamma}$ (where $\gamma \approx 3$ in the study), far outperforming the theoretical $1/N$ bound because the split pushes the per-stage service below the bottleneck.
The Bottleneck Principle: In a tandem pipeline, only the slowest stage ( $s_{max}$ ) determines the tail. The order of stages does not affect latency, only the physical location of the queue. A cut that does not reduce $s_{max}$ adds hop cost without reducing the tail.

3. Empirical Findings

The Threshold: The "tail onset" occurs between 7 and 8 µs, coinciding with the publisher period. Below this, the 99th percentile latency equals the service time. Above this, the tail grows rapidly (e.g., at 128 µs, the tail is ~40x the service time).
Cause of the Tail: The tail is caused by the timing of transactions (specifically, the surplus of gaps shorter than the service time), not by the packet rate, message count per packet, or packet size.
- Shuffling the order of gaps (destroying runs) removes most of the tail at long service times but leaves it largely intact at short service times.
- Randomizing transaction start times removes the tail entirely.
Stateless vs. Stateful: For stateless stages (e.g., decoding), a dispatch pool of cores performs as well as or better than a serial cut. For stateful stages (e.g., order book), a serial cut is required.

Significance and Claims

The paper claims to provide the first measurement-based design rules for HFT systems that explicitly account for the arrival law of exchange feeds.

Revision of Prior Art: It challenges the "single-thread is always best" dogma, arguing that this rule only holds when service times are below the exchange's publisher period. Above this threshold, the "near-critical" self-exciting nature of the feed makes multi-stage pipelines superior for tail latency reduction.
Design Equation: The author proposes a design equation balancing the hop cost ( $h$ ) against the tail reduction ( $\Delta$ ). The optimal number of stages is determined by the service time of the slowest stage relative to the publisher period.
Practical Guidance:
- Do not split if the service time is below the publisher period (~7.5 µs).
- Do split if the service time exceeds the period, aiming to make the slowest stage shorter than the period.
- Focus on the slowest stage: Optimization efforts should target reducing the service time of the bottleneck stage, not simply increasing the number of threads.
- Measure the threshold: Designers must measure the publisher period of their specific feed, as it is a constant of the exchange's infrastructure, not a function of market volatility.

The paper concludes that while the single-thread rule remains valid for very fast paths, the "multi-actor tandem" is the correct design for slower paths on clustered feeds, provided the split is executed to minimize the slowest stage's service time. The findings are validated against a live production system, confirming that the theoretical tail reduction is achievable in practice.