🤖 machine learning

TraceSynth: Generating Production-Quality Kernel Traces with Constraint-Guided Diffusion Models

TraceSynth is a constraint-guided diffusion framework that generates high-quality synthetic kernel execution traces to cost-effectively augment limited real data for system diagnostics, achieving near-baseline performance on deterministic workloads while significantly reducing computational costs through lightweight multi-channel modeling.

Original authors: Yuvraj Sehgal, Sneh Patel, Mahsa Panahandeh, Naser Ezzati-Jivan, Francois Tetreault

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuvraj Sehgal, Sneh Patel, Mahsa Panahandeh, Naser Ezzati-Jivan, Francois Tetreault

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to fix a giant, complex machine. To do this, you need to show the robot thousands of hours of video footage of the machine running, breaking down, and getting fixed. But here's the catch: filming that footage is incredibly expensive, takes up massive amounts of storage space, and sometimes the machine's owners are too worried about privacy to let you film the sensitive parts. Plus, the machine rarely breaks in the exact same way twice, so you might miss the weird, rare glitches that are actually the most important to learn from.

This is the problem the researchers at Brock University and the University of Ottawa faced. They wanted to build a tool called TraceSynth that could act like a "movie director" for computer systems. Instead of filming the real machine, TraceSynth uses a special kind of AI (called a diffusion model) to imagine and generate brand-new, fake video footage of the machine running. The goal? To give the robot enough practice data to learn how to fix things without ever needing to film the real, expensive, or private machine.

The "Fake News" Problem and the "Safety Net"

You might think, "If the AI just makes things up, won't it create nonsense?" That's a fair worry. If the AI generates a video where a part of the machine disappears or a button is pressed before the machine is even turned on, that's useless.

To fix this, the team added a constraint-guided repair step. Think of this as a strict editor or a safety net. Once the AI generates a fake trace, this editor checks it against the rules of the universe (like "you can't delete a file before you create it"). If the AI makes a mistake, the editor fixes it instantly. The paper shows that this safety net is a game-changer, especially when the AI is working with short snippets of data. It can boost the quality of the fake data by up to 4.3%, ensuring the "fake" traces follow the same logical rules as the real ones.

The Secret Ingredient: Time Matters More Than Details

Here is the most surprising discovery from their experiments. You might assume that to make a perfect fake trace, the AI needs to know everything about the machine: the specific name of every thread, the exact CPU core, the return codes, and the process names.

The paper explicitly argues against this idea. They found that adding all these extra details actually didn't help much. In fact, a "lightweight" AI that only looked at two things—what event happened and how much time passed between events—performed almost just as well as the super-complex AI that knew everything.

The Finding: A simple model using just 2 channels of data (event and time) retained 97–99% of the performance of the full 6-channel model, but at roughly half the computational cost.
The Takeaway: Don't over-engineer the generator. Knowing the order and timing of events is far more important than knowing the specific names of every part.

The Magic Number: 4096

While the AI didn't need more details, it desperately needed more time. The researchers tested how much "context" (how many events in a row) the AI could look at at once.

When the AI looked at only 256 events, the fake data was pretty bad.
When they increased the view to 4096 events, the quality skyrocketed.

This jump in context length resulted in a 104% relative improvement in quality. It's like the difference between trying to understand a movie by looking at a single frame versus watching a whole scene. For the most predictable, math-heavy workloads (like a benchmark called scimark2), the fake data generated with this long view was so good that it achieved an 87.2% F1-Macro score, which is only 2.6 percentage points away from using real data.

When Does This Work? (And When Does It Fail?)

The paper is very clear about where this magic works and where it doesn't.

It Works Great For: Deterministic, compute-heavy tasks. These are like a robot arm painting a car in a factory; the steps are predictable and repeatable. For these, the fake data is a near-perfect substitute for real data.
It Struggles With: Chaotic, I/O-heavy workloads. These are like a busy airport terminal where planes arrive and leave based on unpredictable weather and traffic. The paper found that for these messy, asynchronous tasks (like stream or iozone), the fake data still had significant gaps, with performance dropping by 25–36% compared to real data. The AI simply couldn't guess the complex, random interactions happening outside the system.

The Verdict

TraceSynth isn't a magic wand that solves every problem, but it is a powerful new tool for specific situations. The authors suggest that for industries trying to train AI to monitor their systems, they can now:

Save money and privacy by using these generated traces instead of collecting massive amounts of real, sensitive data.
Focus on time, not details, by using simpler models that look at long sequences of events rather than trying to memorize every single attribute.
Use a safety net (the constraint repair) to ensure the fake data makes logical sense.

However, the paper warns that for highly chaotic systems, we shouldn't rely on this fake data alone yet. It's a fantastic way to "stress-test" systems and fill in the gaps for predictable workloads, but for the messy, real-world chaos of heavy input/output, we still need to be careful. The researchers are now looking to test this on their own industrial networks to see if it holds up in the wild, but for now, the results suggest that with the right context length and a good safety net, we can teach our robots using stories we made up ourselves.

Technical Summary: TraceSynth

Problem Statement

Modern production systems increasingly rely on machine learning (ML) for operational intelligence, such as predicting bottlenecks and reducing mean time to resolution (MTTR). These models require high-fidelity kernel-level execution traces to capture fine-grained internal states (scheduler decisions, memory allocations, I/O operations) rather than just aggregate metrics. However, collecting production traces is hindered by three primary constraints:

Runtime Overhead: Continuous instrumentation (e.g., via LTTng) introduces significant performance penalties (typically 1.5–1.6×), making it impractical for latency-sensitive services.
Privacy and Compliance: Traces often contain sensitive data (file paths, network endpoints) that violate data retention policies, preventing the aggregation of large-scale datasets for centralized training.
Data Scarcity: Real traces rarely capture "long-tail" failure modes or diverse execution patterns necessary for robust model training.

Existing synthetic generation approaches (e.g., Markov chains, rule-based generators, or GANs) fail to capture long-range temporal dependencies, multi-attribute correlations, or structural validity required for kernel traces.

Methodology: TraceSynth

The authors propose TraceSynth, a framework based on Denoising Diffusion Probabilistic Models (DDPMs) to generate synthetic kernel traces that augment limited real data. The framework operates through four stages:

1. Data Preprocessing

Raw kernel traces (decoded from LTTng) are transformed into a structured, multi-channel representation. Each event is encoded across six channels:

Event: Event type identifier.
dt: Inter-event time delta (log-normalized).
cpu: CPU core identifier.
tid: Thread identifier (hashed).
comm: Process command name.
ret: System call return value.

Traces are segmented into overlapping fixed-length windows ( $L \in \{256, 1024, 4096\}$ ) to create training samples.

2. Diffusion Model Architecture

TraceSynth employs a Transformer-based denoising diffusion model.

Input: Multi-channel sequences where discrete channels are embedded via learned tables and the continuous time-delta channel is projected via an MLP.
Forward Process: The model progressively corrupts the latent representation with Gaussian noise over $T=1000$ steps.
Reverse Process: A Transformer encoder predicts the injected noise. The training objective combines a standard diffusion loss with an auxiliary reconstruction loss to ensure semantic consistency across channels.

3. Constraint-Guided Repair

To address the issue of generative models producing locally plausible but globally invalid sequences, TraceSynth introduces a post-hoc repair mechanism.

Constraint Learning: Instead of hard-coded rules, the system mines invariants directly from real traces, including event transition graphs, temporal bounds, CPU affinity rules, and attribute validity sets.
Repair: Detected violations (e.g., invalid event transitions or out-of-bounds timestamps) are corrected by sampling from the learned allowed sets conditioned on the event type. This decouples generation from validation.

4. Evaluation Strategy

Synthetic data quality is evaluated via a downstream next-event prediction task (predicting the next event type given 128 preceding events). The study compares models trained on:

Real data only.
A 50/50 mix of real and unrepaired synthetic data.
A 50/50 mix of real and constraint-repaired synthetic data.

Key Contributions

Problem Characterization: Defined the specific data scarcity challenges for ML-based diagnostics in industrial settings (specifically citing Ciena's constraints).
TraceSynth Framework: Proposed a multi-channel diffusion model capable of synthesizing up to 4096-length sequences across six trace attributes, coupled with a constraint-learning module for invariant preservation.
Empirical Evaluation: Demonstrated that synthetic augmentation can effectively double training data sizes for deterministic workloads with minimal performance degradation.
Practical Guidance: Provided insights into cost-quality trade-offs, showing that lightweight models often suffice for industrial use cases.

Experimental Results

The evaluation utilized six benchmarks from the Phoronix Test Suite (e.g., scimark2, ffmpeg, iozone) representing compute-heavy, I/O-heavy, and mixed workloads.

Workload Dependence: Performance is highly dependent on workload type.
- Deterministic/Compute-heavy: For scimark2, synthetic augmentation at context length $L=4096$ achieved an 87.2% Macro-F1, only 2.6 percentage points below the real-only baseline (89.8%).
- I/O-Heavy: Workloads like stream and iozone showed significant degradation (25–36% drop), indicating that synthetic traces struggle with asynchronous I/O and complex kernel interactions.
Context Length: Increasing context length is the dominant factor in quality. Moving from $L=256$ to $L=4096$ yielded a +104% relative improvement in Macro-F1.
Constraint-Guided Repair: Repair consistently improved data quality, particularly at shorter context lengths (up to +4.3% improvement for scimark2 at $L=256$ ). At $L=4096$ , the marginal benefit diminished as the model implicitly learned more invariants.
Model Complexity (Ablation): Surprisingly, lightweight 2-channel models (Event + Time) retained 97–99% of the performance of full 6-channel models while reducing computational cost. Feature-rich models offered negligible gains except for highly structured workloads.

Significance and Claims

The paper claims that TraceSynth offers a cost-effective solution for augmenting kernel traces in production observability pipelines. Its significance lies in:

Feasibility: Proving that diffusion models can generate structurally valid, production-quality traces for deterministic workloads, reducing reliance on costly real-world data collection.
Safety: The constraint-guided repair mechanism acts as a "safety net," ensuring generated traces respect system invariants, making them safer for downstream training and stress-testing.
Efficiency: The finding that simple 2-channel models are often sufficient challenges the assumption that complex multi-attribute modeling is always necessary, allowing for lower training costs and inference latency.
Practical Application: The framework enables organizations to stress-test trace-driven infrastructure against edge cases and facilitates privacy-preserving development by sharing synthetic, statistically similar data instead of sensitive production logs.

The authors maintain a modest stance, acknowledging that while the approach works well for structured workloads, highly asynchronous I/O systems remain challenging and require further validation on proprietary production traces.