nlm: Real-Time Non-linear Modal Synthesis in Max

This paper introduces \texttt{nlm}, an open-source set of C++ Max externals that enables efficient, real-time non-linear modal synthesis for strings, membranes, and plates, thereby making advanced physical modeling accessible to composers and sound designers through interactive parameter control and multichannel output.

The Gist

Imagine you are a sound designer or a musician trying to create the perfect sound of a guitar string snapping, a drum skin vibrating, or a metal plate ringing out. Traditionally, to get these sounds, you might just play back a recording. But what if you want to change the sound while it's happening? What if you want to make the string thicker, the drum skin tighter, or the plate made of a different metal, all in real-time?

This is where Physical Modelling comes in. Instead of playing a recording, you build a virtual "mathematical instrument" that simulates how real objects move and vibrate.

The paper introduces a new tool called nlm (Non-linear Modal Synthesis), which is a set of plugins for a popular music software called Max. Here is how it works, explained through simple analogies:

1. The "Lego" Analogy: Breaking Sound into Modes

Imagine a complex sound, like a bell ringing, isn't just one single noise. Instead, think of it as a giant tower built out of Lego bricks.

• Linear Synthesis (The Old Way): In the past, these Lego bricks (called "modes") were static. If you hit the bell, the bricks vibrated, but they didn't really talk to each other. They just shook independently. This is fine for simple sounds, but real instruments are messy. When you pluck a guitar string hard, the string stretches, changes its tension, and the pitch shifts slightly. The bricks need to interact.
• Non-Linear Synthesis (The New Way): The nlm tool allows these Lego bricks to talk to each other. When one brick moves, it pushes or pulls on its neighbors. This creates the "squishy," complex, and realistic sounds of real-world objects where tension and shape change as they vibrate.

2. The "Virtual Workshop": What nlm Does

The authors built nlm to be a "workshop" inside your computer where you can build these vibrating objects.

• The Objects: You can build virtual strings (like a guitar), membranes (like a drum skin), or plates (like a cymbal or a metal sheet).
• The Controls: Instead of writing complex code, you get a simple dashboard. You can turn knobs to change:
  • Tension: How tight is the string?
  • Thickness: Is the plate made of thin aluminum or thick steel?
  • Damping: Does the sound ring out forever or stop quickly?
• The Magic: Because the math is optimized (using a fast "calculator" library called Eigen), you can turn these knobs while the sound is playing, and the sound changes instantly and realistically.

3. The "Remote Control" for Sound

One of the coolest features is how you "hit" these virtual objects.

• The Problem: Usually, to make a sound, you need a hammer hitting a drum. In a computer, simulating that hammer hitting the drum and the drum pushing back on the hammer is very hard math.
• The nlm Solution: They simplified it. You can send a "force signal" (like a burst of noise or a musical note) to a specific spot on your virtual drum.
• Creative Twist: You don't even have to use a drumstick! You can use a microphone to record your voice, filter it, and use that as the "hit." This means you can "play" a virtual metal plate with your voice, creating sounds that are impossible in the real world.

4. Why This Matters

Before this tool, if you wanted to simulate a complex, non-linear vibrating plate, you had to be a math genius and write hours of code, or run slow simulations that took minutes to finish (not real-time).

nlm lowers the barrier. It puts this super-powerful physics engine into a box that musicians and sound designers can open and play with immediately.

• For Composers: You can create instruments that don't exist in nature.
• For Sound Designers: You can make sci-fi sound effects that feel physically real.
• For Performers: You can control the physics of the sound with your hands in real-time.

The Catch (Limitations)

Like any powerful engine, it has limits:

• Stability: If you hit the virtual object too hard, the math can get confused and the sound might glitch (like a digital crash). The authors are working on a "safety valve" to prevent this.
• Computer Power: Simulating hundreds of Lego bricks interacting with each other takes a lot of brainpower. If you try to simulate a massive, complex object with too many details, your computer might slow down. However, for most standard uses, it runs smoothly.

Summary

nlm is like giving a musician a virtual physics lab. Instead of just playing back a recording of a drum, you get to build the drum, change its material, hit it with a laser beam or a voice, and watch how the physics of the universe react in real-time. It makes the impossible sounds of the future available to anyone with a computer.

Technical Summary

Here is a detailed technical summary of the paper "nlm: Real-Time Non-linear Modal Synthesis in Max" by Rodrigo Diaz, Rodrigo Constanzo, and Mark Sandler.

1. Problem Statement

Physical modelling synthesis is a powerful technique for generating realistic and complex sounds by simulating the physical properties of objects. While modal methods (decomposing systems into resonant modes) are highly efficient and well-suited for real-time environments like Max and Pure Data (Pd), existing implementations in these environments have largely been limited to linear models.

Although non-linear modal models for strings, membranes, and plates have been extensively studied and implemented in offline simulations (using MATLAB or C++), their deployment in real-time interactive audio environments has been limited. Existing Max/Pd tools (e.g., Modalys, SDT) primarily support linear synthesis or lack full integration of complex non-linear behaviors (such as tension-induced non-linearities in strings or von Kármán coupling in plates). There is a need for a unified, computationally efficient, and interactive real-time implementation of these non-linear models to lower the barrier for composers and sound designers.

2. Methodology

The authors developed nlm, a set of Max externals implemented in C++ that realizes non-linear modal synthesis. The methodology involves:

• Mathematical Foundation: The system is based on the general scalar partial differential equation governing the transverse displacement ($w$) of non-linear strings, membranes, and plates. This equation accounts for material density ($\rho$), damping ($d_1, d_3$), bending stiffness ($D$), initial tension ($T_0$), and non-linear forces ($f_{nl}$).
  • Strings: Modeled using the Kirchhoff-Carrier approximation (tension changes due to deformation).
  • Membranes: Modeled using the Berger approximation.
  • Plates: Modeled using the von Kármán equations, which involve coupling between transverse and in-plane modes.
• Modal Decomposition: The displacement is decomposed into a sum of $M$ modes ($q_\mu$). The system is transformed from partial differential equations into a set of coupled ordinary differential equations in modal coordinates.
• Discretization: The authors use an impulse-invariant discretization to derive an explicit update scheme for the modal coordinates. This allows for efficient time-stepping.
  • The update scheme involves coefficient vectors ($a_1, a_2, b_1$) derived from the sampling period and modal frequencies.
  • Non-linear terms are calculated as functions of the modal coordinates, involving coupling coefficients (tensors) that depend on the geometry and material properties.
• Implementation Architecture:
  • Core Library: Written in C++ using the Eigen library for optimized matrix operations.
  • Externals: Four specific Max objects are provided: nlm.string~, nlm.plate~, and their multi-channel counterparts mcs.nlm.string~ and mcs.nlm.plate~.
  • Geometry Handling: The system natively supports rectangular geometries with simply supported boundary conditions. However, it allows users to load custom eigenmodes, eigenvalues, and coupling coefficients computed externally, enabling the simulation of complex geometries or alternative boundary conditions.
• Interaction Model: The current implementation assumes the external excitation force is independent of the resonator's state (displacement/velocity). This simplifies the interaction, allowing users to apply force profiles (envelopes, noise, live inputs) directly. A hybrid excitation method is also demonstrated, combining filtered noise with live recordings.

3. Key Contributions

1. Unified Real-Time Implementation: The first unified set of Max externals providing efficient, real-time non-linear modal synthesis for strings, membranes, and plates.
2. Computational Efficiency: By leveraging the Eigen C++ library and optimized update schemes, the system achieves performance suitable for interactive audio, handling roughly 100 modes for plates and several hundred for strings/membranes on current hardware.
3. Flexibility and Customization:
   • Interactive Control: All physical parameters (Young's modulus, density, tension, damping, etc.) are exposed as interactive attributes.
   • Custom Modal Data: Users can load pre-computed modal data and coupling matrices, bypassing the limitation of native rectangular geometry support.
   • Multichannel Output: Support for multiple readout positions, essential for spatial audio applications.
4. Open Source Availability: The software is released as open-source under the Creative Commons Attribution License, hosted on GitHub.

4. Results and Performance

• Functionality: The externals successfully generate non-linear sounds characteristic of real-world instruments (e.g., inharmonicity in strings, complex collision sounds in plates).
• Stability:
  • The linear modal filterbank is unconditionally BIBO-stable due to the impulse-invariant transform.
  • The non-linear coupled system is conditionally stable. The authors enforce a sampling rate condition ($f_s > \omega_{max}/2$) to maintain stability under typical performance conditions.
  • Limitation: Extremely strong excitations can inject enough energy to cause numerical instability (audio clicks).
• Computational Load: Complexity scales as $O(NM^2)$ for the von Kármán plate model (where $N$ is the number of in-plane modes and $M$ is the number of transverse modes). While manageable for moderate mode counts, high mode counts can overload the CPU.

5. Significance and Future Work

Significance:
The nlm package significantly lowers the barrier for musicians, composers, and sound designers to explore non-linear physical modelling. By integrating these advanced models into the familiar Max environment, it bridges the gap between theoretical acoustic research and practical creative application. It enables the exploration of expressive potentials (e.g., tension-induced pitch shifts, complex collision dynamics) that are impossible with linear synthesis.

Future Work:
The authors outline several planned improvements:

1. Robust Integration: Implementing more robust integration schemes (e.g., energy-clamping strategies) to prevent numerical instability during extreme excitations.
2. Non-linear Contact Models: Incorporating non-linear contact models (similar to the SDT package) to simulate dynamic feedback between the excitation and the resonator state.
3. Native Geometry Support: Providing native support for arbitrary geometries within the externals to eliminate the need for external preprocessing of modal data.
4. Pd Port: Releasing a version of the externals for Pure Data (Pd).

In conclusion, nlm represents a major step forward in making sophisticated non-linear physical modelling accessible for real-time creative use, offering a flexible, extensible, and efficient toolset for the next generation of sound design and musical composition.