⚡ electrical engineering

Supercharging Packet-level Network Simulation of Large Model Training via Memoization and Fast-Forwarding

Wormhole is a user-transparent packet-level network simulation kernel that achieves massive speedups (up to 1012x) for large model training simulations by using memoization and fast-forwarding to skip redundant, steady-state traffic patterns while maintaining high fidelity.

Original authors: Fei Long, Kaihui Gao, Li Chen, Dan Li, Yiwei Zhang, Fei Gui, Yitao Xing, Wenjia Wei, Bingyang Liu

Published 2026-02-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Fei Long, Kaihui Gao, Li Chen, Dan Li, Yiwei Zhang, Fei Gui, Yitao Xing, Wenjia Wei, Bingyang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a director trying to film a massive, epic movie about a busy city. To make it look realistic, you want to film every single person walking, every car driving, and every raindrop falling. This is "Packet-level Simulation"—it is incredibly detailed and accurate, but it is a nightmare to film. If you tried to film a whole year of city life one second at a time, it would take you a thousand years to finish the movie.

In the world of Artificial Intelligence, engineers need to "film" how data (packets) moves through massive supercomputers to train models like GPT. Currently, this "filming" is so slow that simulating one training session can take weeks.

This paper introduces Wormhole, a tool that acts like a "Smart Director" to speed up this process without losing the realism. Here is how it works using three clever tricks:

1. The "Deja Vu" Trick (Memoization)

Imagine you are filming a scene where a specific group of cars gets stuck at a red light, honks, and then moves on. If you have to film this exact same scene 1,000 times, you don't need to actually set up the cameras, the actors, and the lights every single time.

Instead, you film it once, save it in a "clip library," and whenever that same situation happens again, you just play the clip.

In the paper: The researchers noticed that in AI training, data traffic often hits the same "traffic jams" (contention patterns) over and over. Wormhole recognizes these patterns, looks them up in a database, and "replays" the result instead of recalculating it from scratch.

2. The "Fast-Forward" Trick (Steady-State Skipping)

Imagine you are filming a highway. After the initial rush hour, the cars settle into a steady, predictable flow where they all drive at 60 mph. There’s no point in filming every single car passing a certain point for three hours; it’s boring and repetitive.

Instead, you can just say, "Okay, the traffic is steady now. Let's skip ahead three hours to when the next big accident happens."

In the paper: Once the data flows stabilize and reach a "steady state," the math becomes predictable. Wormhole detects this stability and "fast-forwards" the simulation clock, skipping the millions of tiny, redundant events that don't change the big picture.

3. The "Neighborhood" Trick (Network Partitioning)

If you are trying to manage a massive city, you don't try to watch every street at once. You divide the city into neighborhoods (like Brooklyn, Queens, and Manhattan). If a parade is happening in Brooklyn, it doesn't affect the traffic in Manhattan. You can study them separately.

In the paper: Wormhole divides the massive supercomputer network into smaller "neighborhoods" (partitions). This allows the simulator to focus on one area at a time and prevents a "traffic jam" in one part of the computer from slowing down the math for the entire system.

The Result: From Weeks to Minutes

By using these tricks, the researchers achieved something incredible.

Before Wormhole, simulating a large AI training session might take 9 hours. With Wormhole, it takes only 5 minutes. That is a 1,000x speedup!

And the best part? Even though they skipped so much "filming," the final "movie" is still 99% accurate. It’s like watching a movie that uses clever editing to skip the boring parts but still tells the exact same story.

Technical Summary: Supercharging Packet-level Network Simulation of Large Model Training via Memoization and Fast-Forwarding

1. Problem Statement

As Large Language Model (LLM) training scales to clusters containing $10^3$ to $10^6$ GPUs, the demand for high-fidelity network simulation has surged. Packet-level Discrete-Event Simulation (PLDES), such as ns-3, is the gold standard for capturing critical performance events like queueing, packet loss, and congestion control dynamics. However, PLDES suffers from extreme computational inefficiency: simulating the massive number of discrete events ( $>O(10^{12})$ ) generated by GB-level "elephant flows" can take weeks for a single training iteration.

Existing optimization strategies face a trade-off:

Coarse-grained/Flow-level models: Offer speed but suffer from high error margins (10%–25%) because they ignore transient packet-level dynamics.
Parallelization (Multi-core/Distributed DES): Offers speedup but hits a sub-linear scaling upper bound due to synchronization overhead.

2. Key Insight

The authors observe that LLM training traffic is highly repetitive and predictable. Specifically, two types of computational redundancy exist:

Repeated Contention Patterns (Unsteady-states): Identical traffic conflicts (e.g., incast or core switch contention) occur repeatedly under fixed parallelization strategies, leading to reproducible rate evolution dynamics.
Steady-states: After Congestion Control Algorithms (CCAs) converge, flows enter a regime where transmission rates and queue lengths stabilize, making the step-by-step simulation of every packet redundant.

3. Methodology: The Wormhole Kernel

The authors propose Wormhole, a user-transparent PLDES kernel that integrates into existing simulators (like ns-3) to automate the skipping of these redundant periods. Its architecture relies on three core pillars:

Port-level Network Partitioning: To prevent the entire network from being bottlenecked by a single non-stable flow, Wormhole divides the network into independent "partitions" based on shared ports. This allows the simulator to identify steady-states in specific sub-networks without waiting for the entire global network to stabilize.
Memoization for Unsteady-states: To handle transient phases, Wormhole uses a Flow Conflict Graph (FCG). The FCG abstracts the structural (which flows overlap) and quantitative (instantaneous rates) features of a partition. When a new unsteady-state occurs, Wormhole queries a database; if a matching FCG is found, it reuses the historical results (convergence time and volume) to "jump" to the end of the transient phase.
Fast-forwarding for Steady-states: Wormhole employs a rate-based identification algorithm. By monitoring the fluctuation of the sending rate ( $R$ ) within a window, it identifies when a flow has converged. Once a partition is in a steady-state, Wormhole "pauses" the packets at the switch ports (maintaining constant buffer occupancy to preserve fidelity) and fast-forwards the simulation clock to the next disruptive event (e.g., a new flow entering or an existing flow exiting).

4. Key Contributions

Novel Optimization Paradigm: The first to introduce the concepts of memoization and fast-forwarding specifically for packet-level network simulation.
Theoretical Framework: Provides mathematical proofs (Theorems 1–3) showing that the errors introduced by estimating steady-state rates and durations are bounded and controllable.
Algorithmic Innovations: Developed a port-level partitioning algorithm, a weighted graph-isomorphism-based memoization lookup, and a robust steady-state detection mechanism.
Seamless Integration: Designed to be orthogonal to existing multi-threading technologies (like Unison), allowing for compound acceleration.

5. Results and Evaluation

The evaluation used various LLM workloads (GPT-style and MoE) across different scales (64 to 1024 GPUs):

Speedup: Wormhole achieved up to 744× speedup over original ns-3. When combined with Unison (parallelization), it achieved a massive 1012× speedup, reducing a 9-hour simulation to just 5 minutes.
Accuracy: Maintained a bounded error of <1% in Flow Completion Time (FCT), significantly outperforming flow-level simulators (~20% error).
Robustness: Demonstrated high insensitivity to hyper-parameter selection ( $\theta, l$ ) and various network topologies (Fat-tree, Clos).
Real-world Validation: Tested against real-world traces from NVIDIA A100 clusters, showing comparable accuracy to industry-standard tools like ASTRA-sim but with vastly superior speed.

6. Significance

Wormhole bridges the gap between high-fidelity packet-level simulation and the massive scale required for modern AI infrastructure. By mathematically proving that transient and steady-state behaviors can be bypassed without losing accuracy, it enables researchers and engineers to explore large-scale LLM network designs (topologies, protocols, and orchestration) in minutes rather than weeks, significantly accelerating the development cycle of generative AI systems.