⚡ electrical engineering

Compiling Differentiable Audio Graphs to Real-Time DSP

This paper introduces ADAC, a compiler that automatically translates trained differentiable audio models into efficient, stable, real-time FAUST plugins by generating framework-agnostic intermediate representations and verifying stability through impulse response matching and certificate checks.

Original authors: Facundo Franchino, Sebastian J. Schlecht

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Facundo Franchino, Sebastian J. Schlecht

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, complex recipe for a new type of sound effect, written in a secret, high-level language that only a specific computer kitchen (a machine learning framework like PyTorch) can understand. You've spent hours tweaking the ingredients (the math) to get the perfect flavor. But here's the problem: you can't actually serve this dish to the public because the restaurant's kitchen (real-time audio software) only speaks a different, older language called FAUST.

Usually, a chef would have to manually rewrite the entire recipe by hand, translating every single instruction. This is slow, prone to typos, and if you change one ingredient later, you have to rewrite the whole thing again.

ADAC is the "automatic translator" that solves this problem. It takes your secret recipe and instantly converts it into the restaurant's language without losing a single drop of flavor.

Here is how the paper explains this process using simple analogies:

1. The Automatic Translator (The Compiler)

Think of the AI model as a complex tree structure. ADAC walks through this tree, picks out the exact ingredients (numbers and connections), and writes them down in a neutral, universal format (JSON). It then uses this format to generate the final code (FAUST).

The Magic: It doesn't just guess; it ensures the new code produces the exact same sound as the original AI model, down to the tiniest mathematical detail. It's like a photocopier that is so perfect the copy looks identical to the original, even under a microscope.

2. The "Live" Kitchen (Audible Training)

Usually, when you train an AI, you wait until the very end to hear the result. ADAC changes the game by making the training process "audible."

The Analogy: Imagine a chef tasting the soup after every single pinch of salt they add, rather than waiting until the pot is full. As the computer learns and adjusts the math, the plugin updates instantly in the background. You can hear the sound effect improve in real-time, step-by-step.

3. The Safety Harness (Stability Certificates)

AI models are often unstable; if you turn a knob too far, the sound might explode into a screeching noise or crash the computer.

The Analogy: Before the final plugin is built, ADAC runs a "safety inspection." It checks a mathematical certificate to ensure the sound effect won't go haywire, no matter how the user turns the knobs. If the math says it's unsafe, the system refuses to build the plugin, acting like a strict building inspector who won't sign off on a shaky bridge.

4. The Master Knobs (Macro-Controls)

The raw numbers inside the AI are messy. If you tried to give a user a slider for every single number, they would break the sound effect.

The Analogy: Instead of giving the user 100 tiny, confusing dials, ADAC gives them three "Master Knobs":
- Reverb Time: How long the echo lasts.
- Dry/Wet: How much of the original sound vs. the echo you hear.
- Pre-delay: How long before the echo starts.
  These knobs are "smart." No matter how you turn them, the system automatically adjusts the hidden math to keep the sound stable and musical.

5. The Universal Adapter (Deployment)

Once the translation is done and the safety check passes, ADAC doesn't just give you one file. It acts like a universal adapter plug.

The Analogy: You press one button, and it instantly packages the sound effect into formats that work on Mac, Windows, web browsers, and even specialized hardware chips. It's like printing a document and having it automatically formatted for a letter, a poster, and a mobile screen all at once.

The Bottom Line

The paper demonstrates this with a specific type of sound effect called a "Feedback Delay Network" (which creates echoes and reverberation). They trained an AI to design a specific echo, and ADAC successfully translated it into a working, safe, real-time plugin in seconds.

The authors claim this isn't just about echoes; the system is built to handle any sound effect that is made of series, parallel, and recursive connections. It bridges the gap between experimental AI research and usable, professional audio tools, ensuring that what you design in the lab is exactly what you get in the final product.

Technical Summary: Compiling Differentiable Audio Graphs to Real-Time DSP

Problem Statement
Differentiable audio processors are increasingly designed and optimized within machine learning frameworks (e.g., PyTorch) using gradient descent. However, a significant deployment gap exists: these optimized models remain confined to their training environments. Converting them into real-time audio effects typically requires manual rewriting in dedicated digital signal processing (DSP) languages like FAUST. This translation process is error-prone, demands rigorous verification, and disconnects research prototypes from production-ready tools. Existing differentiable audio systems, such as DDSP and FLAMO, face similar hurdles where trained models cannot be automatically exported to efficient, hardware-agnostic real-time code.

Methodology: The ADAC Pipeline
The authors present the Automatic Differentiable Audio Compiler (ADAC), a pipeline that bridges this gap by lowering trained models into framework-agnostic intermediate representations and emitting efficient FAUST code. The process consists of three primary stages:

Extraction and Serialization: The compiler traverses the computational graph of a trained model (specifically demonstrated with Feedback Delay Networks, or FDNs). It extracts learned parameters and serializes them into a JSON intermediate representation. Crucially, the extractor handles parametric spaces (e.g., storing orthogonal feedback matrices as skew-symmetric weights or Householder vectors) by serializing the effective parameters applied to the signal while retaining raw weights to allow for lossless reconstruction of the source model.
Code Emission: The JSON representation is lowered to FAUST code. Structural nodes map to FAUST composition operators: Series becomes :, Parallel becomes a shared-input split, and Recursion becomes ~. The emitter handles delay lines by accounting for FAUST's implicit one-sample delay in recursion, adjusting the emitted delay values ( $z^{-(m_i-1)}$ ) to ensure the total loop period matches the source model exactly.
Stability Verification: Before deployment, the system analyzes the representation for stability using a small-gain argument. It evaluates the product of the maximum singular values of frequency responses across the loop and checks poles on recursive filter sections. This analysis runs on the exact parameter values after decimal formatting and single-precision casting, ensuring the certificate reflects the shipped artifact rather than idealized double-precision values.

Key Contributions
The paper outlines three specific contributions to the workflow of differentiable audio:

Audible Training: The optimization loop is made audible by replacing the model in a running plugin after every gradient step. A callback re-emits the model and atomically rewrites a watched .dsp file, allowing a resident plugin to reload and recompile the network in under 10 ms.
Stable Macro-Controls: Raw trained parameters are unsuitable for user interaction as they are jointly optimized and may destabilize the processor if moved in isolation. ADAC overlays a fixed vocabulary of macro-controls (reverberation time, dry/wet balance, pre-delay) that maintain processor stability at any setting.
Stability Certificates: The exported plugin carries a JSON stability certificate computed from the shipped parameters. The plugin exporter refuses to build a model if the verdict is unstable or unproven, unless explicitly overridden.

Results

Fidelity: The compiled FAUST code matches the source model's impulse response to within floating-point arithmetic noise (residuals at or below $-80$ dB of the peak, consistent with single-precision limits). For a stereo FDN with a direct path, the agreement is within $7 \times 10^{-5}$ of the peak with no alignment applied.
Performance: Benchmarked on an Apple M2 at 48 kHz, the computational cost follows the predicted $\Theta(N^2)$ complexity for dense FDNs until the feedback matrix exceeds the cache. A 32-line FDN runs at approximately 90 times real-time, and a 64-line FDN at 14 times real-time, leaving ample headroom on commodity hardware.
Control Accuracy: A reverberation time control, calibrated in seconds, successfully imposes a homogeneous decay rate. A setting of 0.5 seconds yields a measured reverberation time of 0.5000 seconds.
Deployment: The system successfully compiles a trained FDN into a working plugin (CLAP, VST3, AU) in a few minutes, with macro-controls exposed as automatable parameters.

Significance and Scope
The authors position ADAC not merely as a code generator but as a complete workflow solution that decouples source traversal from target emission. While the case study focuses on FDNs, the architecture is designed to handle any audio graph built from series, parallel, and recursive compositions with parameterized leaves. The paper notes that the current stability guarantees and equivalence arguments are tied to the linear, time-invariant (LTI) setting. Future work involves extending the emitter to nonlinear and time-varying processors (e.g., waveshapers, envelope followers) and carrying the stability guarantees into those domains. The system also supports fixed-point targets via the Syfala flow, which re-quantizes coefficients and recomputes the stability certificate for the target word length.