How Much AI Is in This Track? Quantifying the Proportion of AI-Generated Stems in Hybrid Music Mixtures

Technical Summary: Quantifying the Proportion of AI-Generated Stems in Hybrid Music Mixtures

Problem Statement
Current AI music detection systems operate under a binary paradigm, classifying tracks as either fully human-produced or fully AI-generated. While these systems achieve high accuracy (>99%) on pure tracks by exploiting spectral artifacts (specifically "fakeprints" or periodic peaks) introduced by the transposed convolution layers in neural audio codec decoders, they fail to address the reality of modern music production. Producers increasingly create hybrid mixtures, integrating AI-generated stems (e.g., synthetic drums or basslines) alongside human-performed instruments. In such scenarios, binary detectors are ill-equipped to quantify the proportion of AI content, and their behavior on mixed signals remains uncharacterized. This paper addresses the need for a detection framework capable of estimating the continuous proportion of AI-generated energy within a track, aligning with transparency obligations such as those in the EU AI Act.

Methodology
The authors propose a methodology to construct controlled hybrid mixtures with a known "AI energy ratio" ($\alpha \in [0, 1]$), defined as the fraction of total acoustic energy contributed by AI-generated stems. The approach relies on three core components:

1. AI Reconstruction via Neural Codecs: Instead of using generative models, the study reconstructs human-performed stems using the EnCodec neural audio codec at 3 kbps. This encode-decode process preserves the original musical content while imprinting the specific architectural artifacts (spectral peaks) that binary detectors are trained to recognize. This isolates the detection problem to codec artifacts, removing confounds related to musical style or content differences.
2. Combinatorial Stem Replacement: Using the MoisesDB multi-track dataset (240 professionally produced tracks with 2–10 stems), the authors generate all possible $2^n$ configurations of real and AI-reconstructed stems for each track. This ensures dense sampling of $\alpha$ values across the full $[0, 1]$ range.
3. Regression Formulation: The detection task is reformulated from binary classification to regression. The goal is to predict the continuous value $\alpha$ given a mixed audio signal $x_{mix}$.

Key Contributions

• Reformulation of Detection: The paper shifts the detection objective from binary classification ($y \in \{0, 1\}$) to regression ($\hat{\alpha} \in [0, 1]$), providing a more informative metric for hybrid workflows.
• Controlled Dataset Generation: A reproducible Python library (ARIA) is released to generate hybrid mixtures from any multi-track dataset using any neural codec, enabling systematic study of mixed-content detection.
• Characterization of Binary Detectors: The authors analyze the behavior of a state-of-the-art binary detector (based on Afchar et al.) when applied to hybrid mixtures.
• Regression Baseline: A CNN-based regression model is trained specifically to estimate $\alpha$, establishing a performance baseline for proportional detection.

Experimental Results
The study evaluates both a reproduced binary detector and a regression model with the same CNN backbone on held-out test mixtures:

• Binary Detector Behavior: When applied to mixed content, the binary detector's output score increases with $\alpha$ but acts as a noisy and miscalibrated estimator. It remains near 0 for $\alpha < 0.5$ and jumps to $\sim0.97$ only when $\alpha > 0.9$, significantly underestimating AI content in the mid-range.
• Instrument Sensitivity: Detection sensitivity is highly dependent on the instrument type and its frequency content. Drums and guitars, which carry strong codec-artifact signatures in the higher frequencies, are reliably detected. In contrast, vocals and bass are less detectable; their spectral energy overlaps with the artifacts less distinctly, causing significant distribution overlap between real and AI conditions in binary classification. However, the regression model can extract proportional information from these weaker signals where the binary detector fails.
• Regression Performance: The regression model trained directly on $\alpha$ achieves a Mean Absolute Error (MAE) of 0.076 and an $R^2$ of 0.85 on held-out mixtures (using 5-second windows). In comparison, using the binary detector's raw probability as an implicit $\alpha$ estimate yields an MAE of 0.289 and a negative $R^2$, indicating it performs worse than predicting the mean.
• Window Length: The regression model benefits from longer analysis windows (5s vs. 1s), achieving more stable estimates, whereas the binary detector's incidental correlation with $\alpha$ degrades with longer windows.

Significance and Claims
The paper claims that the regression formulation represents an "initial promising step" toward AI music detection in realistic production workflows. The authors emphasize that their findings are based on codec reconstructions rather than direct generator outputs, meaning the results specifically measure the proportion of codec-generated energy under controlled conditions.

The study concludes that:

1. Hybrid mixtures contain proportional information about AI content that binary classifiers capture incidentally but fail to utilize effectively.
2. A dedicated regression approach significantly outperforms repurposed binary detectors for quantifying AI energy ratios.
3. Detection sensitivity is not uniform across instruments; it is rooted in the frequency distribution of codec artifacts relative to the instrument's spectral content.

The authors position this work as a methodological foundation rather than a deployment-ready tool. They note that future work requires datasets pairing human stems with actual commercial generator outputs (e.g., Suno, Udio) and accounting for professional post-processing (EQ, compression) which may alter artifact signatures. The proposed methodology and library are designed to be codec- and corpus-agnostic, facilitating future research as such data becomes available.