Technical Summary: Parameter-efficient Dual-encoder Architecture with Differentiable Choquet Integral Fusion for Underwater Acoustic Classification

Problem Statement

Underwater acoustic classification is critical for applications ranging from maritime security to environmental conservation. However, the acoustic environment is characterized by stochastic distortions, including multipath interference, frequency-dependent attenuation, and ambient noise. Traditional methods relying on hand-crafted features (e.g., LOFAR, DEMON) struggle with robustness under these varying conditions. While deep learning has automated feature extraction, single-representation approaches face inherent limitations:

• Spectrograms: Model harmonic dependencies well but may filter out relevant acoustic features during time-frequency transformation.
• Waveforms: Preserve fine-grained phase dynamics and broadband events (e.g., cavitation) but are often noisy and complex, making them difficult for models to process directly.

Furthermore, deploying large-scale foundation models for these tasks is computationally expensive and prone to overfitting due to the limited availability of labeled underwater acoustic data. Fully fine-tuning these models is often impractical.

Methodology

The paper proposes a parameter-efficient dual-encoder neural architecture that simultaneously processes raw acoustic waveforms and log-mel spectrograms. The framework consists of three primary components:

1. Dual-Encoder Backbone with PEFT

The system utilizes two parallel branches:

• Waveform Encoder: Initialized with the AVES-nonbio (Animal Vocalization Encoder based on Self-Supervision) architecture, selected for its suitability with non-biological, mechanical ship signatures.
• Spectrogram Encoder: Initialized with the Audio Spectrogram Transformer (AST), pre-trained on AudioSet and ImageNet.

To enable domain adaptation without full fine-tuning, the pre-trained backbone parameters are frozen. Instead, Parameter-Efficient Fine-Tuning (PEFT) modules are injected. The study evaluates two specific PEFT strategies:

• Low-Rank Adaptation (LoRA): Tested with query ranks $r \in \{16, 32\}$.
• Histogram-based Parameter-efficient Tuning (HPT): Tested with histogram bin counts $b \in \{16, 32, 64\}$.

Each encoder maintains distinct, independent sets of fine-tuning parameters shared across its internal layers, regularizing the adaptation process.

2. Differentiable Choquet Integral Fusion

A novel decision-level fusion mechanism is introduced to combine the outputs of the two branches. Unlike static fusion methods (e.g., averaging), this approach uses the Choquet integral to model non-additive interactions (redundancy or synergy) between the waveform and spectrogram representations.

To make the Choquet integral differentiable for end-to-end training, the authors propose a soft-sort gating mechanism:

• Learnable fuzzy measures ($g_w$ and $g_s$) are optimized per class to quantify the independent contribution and significance of each representation.
• A soft-sort gate vector ($s$) is computed using a sigmoid function to conditionally route predictions based on the relative confidence of the waveform ($p_w$) and spectrogram ($p_s$) branches.
• The fused distribution is calculated as a continuous approximation of the Choquet integral, allowing the network to dynamically shift attention to the representation least corrupted by environmental distortions.

3. Optimization Strategy

The model is trained using a composite multi-objective loss function ($L_{total}$):
$$L_{total} = \frac{1}{4} \left( 2 \cdot CE(p_{fused}, y) + CE(p_w, y) + CE(p_s, y) \right)$$
where $CE$ is the cross-entropy loss. The fused prediction is weighted twice as heavily as the individual branch losses to prioritize collaborative decision-making while ensuring each encoder learns discriminative features.

Key Results

The proposed framework was evaluated on the DeepShip (4 classes) and ShipsEar (5 classes + noise) datasets.

• Performance vs. Single Encoders: The dual-encoder architecture consistently outperformed independent single-encoder baselines (both waveform-only and spectrogram-only) across all tuning strategies.
• PEFT Efficiency:
  • Dual-encoder models using PEFT (LoRA and HPT) achieved classification accuracies comparable to, and in some cases exceeding, fully fine-tuned single-encoder state-of-the-art models (e.g., ResNet-50, ConvNeXtV2, SSAST).
  • Specifically, the Dual LoRA and Dual HPT frameworks optimized roughly 100k parameters, yet outperformed the majority of fully fine-tuned single-encoder networks which required significantly more trainable parameters (e.g., ~85k–90k for single encoders, ~175k for dual full fine-tuning).
  • In the DeepShip dataset, the Dual LoRA (r=16) achieved 71.6% accuracy, surpassing the fully fine-tuned AST (72.3%) and AVES (67.9%) when considering the massive reduction in trainable parameters.
• Interpretability: Analysis of the learned fuzzy measures revealed class-specific shifts in reliance. For instance, in the DeepShip "Tanker" class under linear probing, the network assigned higher weight to the waveform (0.22) than the spectrogram (0.18), whereas for "Cargo," it heavily favored the spectrogram. This confirms the model's ability to dynamically select the optimal representation for specific acoustic signatures.
• Robustness: The dual-encoder approach showed superior performance in low false-alarm regions (pAUC0.30), particularly in the ShipsEar dataset where the Dual HPT method statistically outperformed the single-mode full fine-tuning approach.
• Representational Alignment: Centered Kernel Alignment (CKA) analysis indicated that while early layers remained stable, deeper layers diverged more from the fully fine-tuned baseline. The waveform branch exhibited lower similarity to the baseline than the spectrogram branch, suggesting it is a primary bottleneck requiring higher-capacity tuning.

Significance and Claims

The paper claims to present a computationally efficient framework for advanced maritime acoustic recognition that balances performance with resource constraints. Its primary contributions are:

1. Dual-Representation Synergy: Demonstrating that fusing temporal (waveform) and spectral (spectrogram) data yields higher accuracy than either modality alone, effectively mitigating the non-stationary challenges of the underwater channel.
2. Parameter Efficiency: Proving that PEFT modules (LoRA and HPT) can adapt large foundation models to underwater domains with a fraction of the trainable parameters, mitigating overfitting risks on limited datasets.
3. Differentiable Fusion: Introducing a novel, differentiable Choquet integral layer that not only improves accuracy but provides interpretability. By analyzing the learned fuzzy measures, the system reveals how the network dynamically shifts reliance between representations based on class-specific acoustic characteristics and environmental distortions.
4. Operational Advantage: The dynamic gating mechanism allows the system to prioritize the representation least corrupted by local environmental noise (e.g., shifting to waveforms when spectral tonals are masked), offering a robust solution for real-world deployment.

The authors conclude that this framework establishes a viable path for high-performance underwater acoustic classification without the prohibitive computational costs of full network fine-tuning. Future work is proposed to investigate hardware efficiency and quantization methods for the PEFT-integrated backbones.