Robust Generative Audio Quality Assessment: Disentangling Quality from Spurious Correlations

This paper proposes a robust generative audio quality assessment framework that employs domain adversarial training with aspect-specific, data-driven domain definitions to effectively disentangle true perceptual quality from spurious acoustic correlations, thereby significantly improving model generalization and alignment with human ratings.

The Gist

Imagine you are trying to teach a robot how to judge the quality of music or speech. You want it to tell you, "This song sounds amazing!" or "This voice recording is full of static and bad."

The problem is, the robot is a bit of a cheat. Instead of actually listening to the sound, it starts looking for "cheat codes" or shortcuts.

The Problem: The Robot's "Cheat Codes"

Let's say you train your robot using two different libraries of audio:

1. Library A: High-quality classical music recorded in a fancy concert hall.
2. Library B: Rough, low-quality voice notes recorded in a noisy kitchen.

Because the robot is lazy, it quickly learns a spurious correlation (a fake rule): "If the audio sounds like it came from a concert hall, it must be high quality. If it sounds like a kitchen, it must be low quality."

It stops listening to the actual music or voice and just guesses based on the background noise or the recording equipment. This is called learning "spurious correlations."

Now, imagine you give the robot a brand new song recorded in a studio (which sounds like the concert hall) but the song itself is terrible. The robot will say, "Great job!" because it's fooled by the studio sound. It fails because it didn't learn what "good quality" actually means; it just learned to recognize specific types of rooms.

The Solution: The "Blindfold" Training

The authors of this paper propose a clever training method called Domain Adversarial Training (DAT). Think of this as a game of "Hide and Seek" played inside the robot's brain.

1. The Goal: The robot needs to predict the quality score (the "Judge").
2. The Cheat: The robot also has a "Detective" inside it trying to guess where the audio came from (the "Domain").
3. The Twist: The "Judge" is trained to be bad at helping the "Detective."

Every time the robot tries to use a "cheat code" (like "this sounds like Library A"), the "Detective" catches it. The training system then punishes the robot for using that clue. The robot is forced to throw away all the "where did this come from?" information and focus only on the actual sound quality.

The Big Discovery: One Size Does Not Fit All

The most interesting part of this paper is that the authors realized you can't use just one type of "blindfold" for everything. It depends on what you are judging.

They tested two main ways to define the "Domain" (the thing the robot shouldn't look at):

1. The "Source Label" Strategy (The Explicit List)

• How it works: You tell the robot, "Don't look at whether this came from Library A or Library B." You use the actual names of the datasets.
• Best for: Judging Content (e.g., "Is this song enjoyable?" or "Is this story complex?").
• Analogy: Imagine judging a painting. If you tell the artist, "Don't care if the paint was bought at Store A or Store B," they will focus on the art itself. This works well for judging the story or emotion of the audio.

2. The "Clustering" Strategy (The Pattern Finder)

• How it works: Instead of using dataset names, the robot looks at the audio and groups similar-sounding things together automatically (e.g., "All sounds with heavy echo," "All sounds with background traffic").
• Best for: Judging Technical Quality (e.g., "Is there static?" or "Is the voice clear?").
• Analogy: Imagine judging the sharpness of a photo. The "Store A vs. Store B" labels don't matter. What matters is if the photo is blurry or sharp. By grouping photos by "blurry-ness" or "sharp-ness" automatically, the robot learns to ignore the camera brand and focus on the focus.

The Results: A Fairer Judge

When they applied these specific strategies:

• The robot stopped cheating.
• It became much better at ranking new, unseen audio (like AI-generated music it had never heard before).
• It stopped giving high scores just because a recording sounded "fancy."

The Takeaway

In the past, researchers tried to use a single, rigid rule to fix AI audio judges. This paper shows that the best approach is customized.

• If you want to know if a song is fun, tell the AI to ignore the source of the file.
• If you want to know if a voice is clear, tell the AI to ignore the acoustic patterns (like echo or noise) by grouping them smartly.

By teaching the AI to forget where the sound came from and focus only on what the sound is, we get a much more reliable, human-like judge for the future of AI-generated audio.

Technical Summary

1. Problem Statement

The rapid proliferation of AI-Generated Content (AIGC) has created an urgent need for robust automatic metrics to assess perceptual audio quality (Mean Opinion Score, or MOS). However, current automatic MOS prediction models suffer from data scarcity, leading to spurious correlations (shortcut learning).

• The Core Issue: In limited training datasets, models often overfit to dataset-specific acoustic signatures (e.g., specific recording equipment, room reverberation, or background noise) rather than learning generalized quality features.
• Consequence: A model might associate "high quality" with the specific timbre of a dataset's instruments or the ambiance of a recording studio. When deployed on unseen generative scenarios with different acoustic signatures, these models fail to generalize, producing unreliable predictions.
• Gap: Existing solutions often rely on static domain priors or hand-crafted heuristics. There is a lack of systematic investigation into how to define "domains" for adversarial training to best mitigate these biases across different aspects of audio quality.

2. Methodology

The authors propose a Domain Adversarial Training (DAT) framework designed to force the model to learn domain-invariant representations, effectively disentangling true perceptual quality from nuisance factors.

A. Model Architecture

The framework builds upon the MultiGauss backbone (a state-of-the-art MOS predictor) and integrates three key components:

1. SSL Feature Extractor: Uses the pre-trained XLS-R 2B model (frozen) to extract robust, general-purpose audio embeddings.
2. Quality Prediction Backbone (MultiGauss): Predicts a multivariate mean vector ($m$) representing quality scores and a covariance matrix ($\Lambda$) for uncertainty.
3. Domain Discriminator Branch: A parallel branch connected to the shared latent representation ($h$) via a Gradient Reversal Layer (GRL).
   • Mechanism: The discriminator attempts to predict the domain label from $h$. The GRL reverses gradients during backpropagation, forcing the encoder to minimize the discriminator's accuracy. This compels the encoder to strip domain-specific information while retaining quality-relevant features.
   • Loss Function: $L_{total} = L_{task} + \lambda L_{adv}$, where $L_{task}$ is Gaussian Negative Log-Likelihood (GNLL) for MOS prediction, and $L_{adv}$ is cross-entropy for domain classification.

B. Systematic Domain Definition Strategies

The paper's primary innovation is the systematic investigation of three distinct strategies for defining the "domain" ($D$) used in the adversarial branch:

1. DAT-Source (Explicit): Uses metadata (dataset identity, e.g., AudioSet vs. LibriTTS) as domain labels. This targets macro-level production variations.
2. DAT-Kmeans (Implicit/Latent): Uses unsupervised K-means clustering on the latent acoustic embeddings to discover data-driven acoustic patterns (e.g., reverberation, noise profiles) that transcend dataset boundaries. The granularity ($K$) is a tunable hyperparameter.
3. DAT-Random (Control): Assigns random labels to verify if gains come from meaningful disentanglement or merely stochastic regularization.

3. Key Contributions

• Identification of Spurious Correlations: Demonstrated that data scarcity causes models to overfit to acoustic signatures, and proposed DAT as a solution without complex heuristics.
• Aspect-Specific Domain Strategy: Revealed that there is no "one-size-fits-all" domain definition. The optimal strategy depends on the specific MOS aspect being evaluated:
  • Content Attributes (Complexity, Enjoyment): Benefit most from DAT-Source (explicit metadata), as these attributes often correlate strongly with dataset identity.
  • Technical/Functional Attributes (Quality, Usefulness): Benefit most from DAT-Kmeans (latent clustering), as technical degradations (noise, distortion) often overlap across datasets and require fine-grained acoustic texture disentanglement.
• Generalizability: Validated that these findings hold across different backbone architectures (MultiGauss with frozen XLS-R and Audiobox-Aesthetics with fine-tuned WavLM).

4. Experimental Results

The method was evaluated on the AES-Natural dataset, which contains natural recordings (training) and machine-generated audio (evaluation) rated on four dimensions: Production Quality (PQ), Production Complexity (PC), Content Enjoyment (CE), and Content Usefulness (CU).

• Performance Gains:
  • DAT-Source significantly improved PC (MSE reduced from 1.093 to 0.747; SRCC increased to 0.969) and CE, effectively removing dataset-based shortcuts.
  • DAT-Kmeans achieved the best results for PQ (SRCC 0.953) and CU, outperforming explicit source labels in ranking consistency for technical metrics.
  • Both DAT strategies outperformed traditional regularization techniques (L2, High Dropout) and the DAT-Random baseline, proving that gains stem from principled disentanglement, not just noise injection.
• Latent Space Analysis (UMAP):
  • Baseline models showed "domain islands" where samples clustered by dataset identity rather than quality.
  • DAT models collapsed these islands into a unified manifold, creating a continuous "Quality Terrain" where samples from all domains aligned vertically according to their quality scores, confirming the removal of non-causal signatures.
• Granularity Analysis: For DAT-Kmeans, a cluster size of K=8 yielded optimal performance, balancing the capture of fine-grained acoustic textures without over-partitioning the space.

5. Significance

This work addresses a critical bottleneck in the deployment of AIGC quality assessors: robustness to unseen data.

• Practical Impact: By identifying that different quality aspects require different domain definitions, the paper provides a blueprint for building more reliable, generalizable audio quality metrics.
• Theoretical Contribution: It challenges the static view of domain adaptation in audio, showing that implicit, data-driven clustering can be superior to explicit metadata for certain technical tasks.
• Future Direction: The authors propose a unified multi-branch architecture that combines explicit source constraints and latent acoustic clustering to create a universal, robust model for all perceptual dimensions of audio quality.