Technical Summary: GF-DiT: Scheduling Parallelism for Diffusion Transformer Serving

Problem Statement

Diffusion Transformers (DiTs) have become the dominant architecture for modern generative AI, yet existing serving systems struggle to efficiently handle their heterogeneous workloads. Current systems (e.g., vLLM-Omni, SGLang Diffusion) rely on static parallelism, assigning each request a fixed GPU configuration (e.g., a specific sequence-parallel degree) for its entire lifetime. This approach fails to address three critical sources of heterogeneity:

1. Request Heterogeneity: Output resolutions, video durations, and denoising step counts vary significantly, causing orders-of-magnitude differences in computational demand.
2. Stage Heterogeneity: Different stages of the DiT pipeline (encoder, denoising, decoder) exhibit fundamentally different scaling behaviors. For instance, denoising benefits from large sequence-parallel groups, while encoders and decoders are often lightweight and single-rank efficient.
3. System Heterogeneity: Workload dynamics (arrival rates, queue conditions) and service objectives (latency vs. throughput vs. SLO) change over time. A configuration optimal at admission may become inefficient as conditions evolve.

Static parallelism leads to Head-of-Line (HoL) blocking, where long-running requests delay shorter ones, and poor resource utilization, where lightweight stages waste GPU resources or large denoising tasks are under-provisioned. The core problem is that existing systems treat parallelism as a fixed deployment decision rather than a dynamic, schedulable resource.

Methodology

The paper proposes GF-DiT, a policy-programmable runtime that treats GPU parallelism as a first-class schedulable resource. The system dynamically adapts the parallelism of running requests based on workload demands and service objectives. GF-DiT is built upon three key technical pillars:

1. Asynchronous Execution and Reschedulable Trajectory Tasks

GF-DiT decomposes a DiT request into a graph of trajectory tasks (encoding, individual denoising steps, and decoding).

• Rescheduling Boundaries: Unlike autoregressive LLMs where token generation provides fine-grained boundaries, DiT requests expose natural scheduling points at stage and denoising-step boundaries. Completing a step produces a semantically complete latent state, allowing the runtime to safely reassign resources.
• Predictability: DiT execution structures are largely deterministic before execution begins (known resolution, steps, and trajectory). This allows the runtime to estimate execution costs and future resource demands accurately, enabling online adaptation without guessing.
• Asynchronous Model: The runtime separates the control plane (scheduling, dependency tracking) from the execution plane (GPU workers). Dispatch completion is decoupled from device completion, allowing CPU-side scheduling and migration planning to overlap with GPU execution.

2. Group-Free Collectives (GFC)

To make dynamic parallelism practical, the system must support the rapid creation and reconfiguration of execution groups without incurring the high overhead of traditional communication libraries.

• The Challenge: Conventional libraries (e.g., NCCL) require constructing process_group objects for each subgroup. This involves host-side coordination and initialization costs (hundreds of milliseconds) and significant memory overhead, which is prohibitive for fine-grained, online reconfiguration.
• The Solution: GFC eliminates explicit communicator construction.
  • World-Level Setup: A single world-level communication setup is performed at server startup, allocating symmetric communication buffers.
  • Logical Descriptors: Dynamic subgroups are represented by lightweight logical descriptors (rank IDs and ordering) rather than heavy communicator objects.
  • Pairwise-Consistent Ordering: To ensure correctness in overlapping groups, GFC enforces that any pair of ranks observes collective instances in the same order. It uses a double-buffered signaling mechanism attached to rank edges (rather than groups) to manage synchronization tokens, avoiding combinatorial state explosion.
  • Staging: Data movement is handled via staging buffers, allowing the runtime to pipeline local copies with remote transfers and select optimal backends (CUDA kernels, copy engines, TMA) based on message size.

3. Policy-Programmable Runtime

GF-DiT exposes a unified interface where scheduling policies can jointly control task ordering, placement, and parallelism.

• Policy Interface: Policies observe ready tasks, resource availability, and cost estimates, then return dispatch decisions specifying the task and its execution layout (logical group + parallel spec).
• Flexibility: This allows diverse strategies (e.g., FCFS, Shortest Remaining Trajectory First, Earliest Deadline First) to be implemented and evaluated without rewriting the underlying runtime or model logic.
• Simulation: A simulator uses the same task graph and policy interface but replaces GPU execution with cost-model predictions, enabling offline policy exploration and selection.

Key Contributions

1. New Abstraction: The paper introduces the concept of treating GPU parallelism as a first-class schedulable resource, shifting the paradigm from static deployment to dynamic runtime management.
2. GF-DiT Runtime: A policy-programmable system that enables online GPU reallocation through asynchronous execution and reschedulable trajectory tasks.
3. Group-Free Collectives: A lightweight communication abstraction that reduces the overhead of dynamic group formation from hundreds of milliseconds to approximately 60 µs, making elastic parallelism practical.
4. Implementation and Evaluation: A full implementation in vLLM-Omni demonstrating significant performance gains over fixed-pipeline execution.

Results

Evaluated on image (Qwen-Image) and video (Wan2.2) diffusion workloads across H20 and A100 GPUs, GF-DiT demonstrates substantial improvements over static parallelism baselines:

• Throughput: Improved by up to 6.01×.
• Latency: Reduced mean latency by up to 95%.
• SLO Violations: Reduced SLO violation rates by up to 90%.
• Communication Overhead: Reduced communication-group setup overhead from 778 ms to approximately 60 µs.
• Policy Effectiveness: The EDF (Earliest Deadline First) policy achieved the highest SLO attainment (e.g., improving from 7.5% to 88.8% in bursty video workloads), while SRTF (Shortest Remaining Trajectory First) minimized mean latency in specific scenarios. The results confirm that no single static configuration is optimal; dynamic adaptation is necessary.

Significance and Claims

The paper argues that the static nature of parallelism in current DiT serving systems is a fundamental bottleneck. By treating parallelism as a schedulable resource, GF-DiT allows systems to react to workload dynamics and service objectives in real-time.

The authors claim that GF-DiT makes elastic parallelism practical for real-world generative AI serving. The significance lies not in proposing new parallel kernels, but in providing a programmable substrate that allows existing execution layouts to be dynamically selected and reconfigured. This enables a shift from "scheduling requests" to "scheduling parallelism," ensuring that GPU resources continuously flow to where they provide the greatest benefit, thereby simultaneously improving throughput, latency, and SLO attainment. The work suggests that future generative AI serving systems should embrace runtime-managed parallelism as a fundamental systems abstraction.