Quality-Diversity Search in Sound Generation: Investigating Innovation Engines for Audio Exploration

Technical Summary: Quality-Diversity Search in Sound Generation

Problem Statement
Composers and sound designers often face technical barriers when attempting to create and refine tools to achieve specific musical goals. While digital synthesis theoretically allows for the creation of any sound, the vastness of the sonic domain makes manual exploration difficult. Furthermore, the "Meno's Paradox" suggests that discovering truly novel sounds is challenging if one does not already know what to look for; however, serendipitous discovery is possible if new sounds are immediately recognizable upon hearing. Existing methods often rely on human feedback loops, which are limited by human fatigue when evaluating long series of sounds and the difficulty of providing sufficient selection pressure for evolutionary processes. The paper addresses the need to automate the navigation through unexplored sonic territories to generate diverse, high-quality, and innovative sound objects without requiring pre-training on specific datasets or explicit human guidance during the evolutionary loop.

Methodology
The study employs an Innovation Engine architecture, combining a diversity-promoting Quality-Diversity (QD) algorithm with a supervised discriminative model.

• Search Algorithm: The system utilizes MAP-Elites (Multi-dimensional Archive of Phenotypic Elites). To encourage exploration, the algorithm is biased away from niches that produce fewer innovations using a "curiosity score" (a decrementing counter for each niche).
• Sound Synthesis Genome: The evolutionary genome consists of two co-evolving components:
  1. Compositional Pattern Producing Networks (CPPNs): These networks generate patterns over time. In this study, they are restricted to composing periodic functions (sine, square, triangle, sawtooth) to serve as oscillators.
  2. Digital Signal Processing (DSP) Graphs: These graphs process the signals from the CPPNs. They include standard nodes (filters, delays, gain) and custom nodes (wavetable, specialized additive synthesis) to expand the sonic repertoire.
  • The genome evolves from scratch (random initialization) using the NEAT (NeuroEvolution of Augmenting Topologies) algorithm, adding nodes and connections over time.
• Discriminative Model (Reward Signal): A pre-trained YAMNet Deep Neural Network (DNN), trained on the AudioSet dataset, serves as the classifier. It outputs 521 confidence scores corresponding to audio event classes. These scores define the behavior space cells for MAP-Elites and provide the selection pressure.
• Experimental Configurations:
  • Baseline: Co-evolution of CPPNs and DSP graphs.
  • Variants: CPPN-only (no DSP graph), single-cell-win (restricting candidates to win only one cell), and specialized CPPNs (multiple CPPNs instantiated for specific frequency ranges).
  • Temporal Expansion: The behavior space was expanded to include sound durations (0.5s, 1s, 2s, 5s, 10s) as orthogonal dimensions to the classification classes.
  • Single-Objective Control: Runs were conducted where the algorithm optimized for a single class to compare against the multi-objective QD approach.

Key Contributions

1. Novel Synthesis Architecture: The paper introduces a system combining CPPNs and DSP graphs where CPPNs act as specialized signal sources for different frequency ranges. A key finding is that using multiple specialized CPPNs (one per frequency range) results in significantly simpler networks (approx. 1/3 the node count and half the connection count of a single multi-purpose CPPN) while maintaining or slightly improving QD scores.
2. Evolutionary Stepping Stones: The study analyzes "goal switching" (where a champion of one class is the offspring of a champion of another). Results show that lineages frequently traverse unlikely paths, with 20% of goal switches occurring between broad "Musical" and "Non-musical" contexts, demonstrating the system's ability to navigate complex evolutionary landscapes.
3. Temporal Specialization: By expanding the behavior space to include duration, the study reveals that sound objects often specialize within temporal niches. Genomes that are elites at one duration (e.g., 0.5s) rarely remain elites at other durations (e.g., 10s), indicating that the temporal dimension creates distinct evolutionary pressures.
4. Diversity vs. Single Objective: The study demonstrates that while single-objective runs can achieve higher average confidence scores for specific classes, they result in significantly higher genome complexity and lower diversity. The QD approach maintains a balance between performance and structural simplicity.

Results

• Performance: The co-evolution of CPPNs and DSP graphs achieved the highest overall QD scores and subjective aesthetic appeal compared to CPPN-only configurations. The system successfully generated sounds that received high confidence scores across most of the 521 AudioSet classes, though it struggled with specific musical genre classifications (e.g., "Pop music"), likely because the system generates sound materials rather than full compositions.
• Complexity: Single-objective runs led to much higher genome complexity (more CPPN and DSP nodes) compared to QD runs. The specialized multi-CPPN approach reduced complexity without sacrificing performance.
• Duration Effects: Contrary to the hypothesis that longer durations (10s) would allow the classifier to better detect patterns, 0.5s durations yielded higher QD scores in CPPN-only runs. However, when DSP graphs evolved alongside CPPNs, duration had little effect on performance.
• Lineage Analysis: Phylogenetic tracing revealed that elites often descend from parents in different classes, confirming the existence of evolutionary stepping stones that bridge disparate sonic categories.

Significance and Claims
The paper claims that the proposed system is a viable approach for automating the discovery of novel sounds. It argues that diversity-promoting algorithms can bridge the gap between the theoretical realization of sounds and their practical accessibility. The significance of the work lies in:

• Bottom-Up Exploration: Unlike top-down approaches where goals are pre-defined, this method encourages a bottom-up process where the "tool writes the toolmaker," allowing for the discovery of interesting sound objects that can steer creative journeys.
• Scalability: The specialized CPPN architecture scales more gracefully, allowing new frequency ranges or tasks to be added without over-complicating a single network.
• Creative Potential: The generated sound objects are presented not as standalone finished pieces but as materials for further creative endeavors. The authors provide an online explorer and experimental applications (using the sampler AudioStellar) to showcase how these diverse, automated discoveries can inspire human composers and sound designers.
• Modesty: The authors acknowledge that the high-scoring sounds are not always the most realistic representations of their classes (attributed to DNN limitations) and that the system is intended to facilitate or inspire creation rather than replace human intuition. They suggest future work could involve expanding the behavior space beyond predefined classes using dimensionality reduction and unsupervised discovery.