Focus on What Matters: Fisher-Guided Adaptive Multimodal Fusion for Vulnerability Detection

1. Problem Statement

The paper addresses the critical challenge of false negatives in software vulnerability detection, where vulnerable functions slip into production pipelines. While multimodal approaches combining Natural Code Sequences (NCS) (text) and Code Property Graphs (CPG) (structural) have become mainstream, existing fusion methods suffer from two fundamental bottlenecks:

1. Feature Redundancy: Pretrained language models (e.g., CodeBERT) implicitly encode significant syntactic and shallow structural information. Consequently, the explicit structural features from CPG often overlap heavily with NCS representations, leading to redundant signals when simply concatenated.
2. Feature Extraction Asymmetry: There is a performance gap between NCS encoders (strong) and CPG graph encoders (weak). Current fusion mechanisms often mix the high-quality semantic signals of NCS with noisy, insufficiently extracted structural signals from CPG, diluting the discriminative power of the model.

Existing methods (simple concatenation or generic cross-attention) fail to distinguish between task-relevant structural signals and redundant noise, resulting in limited performance gains, particularly in highly imbalanced real-world datasets.

2. Methodology: TaCCS-DFA

The authors propose TaCCS-DFA (Task-Conditioned Complementary Subspace with Dynamic Fisher Attention), a framework that uses Fisher Information to guide the fusion process. The architecture consists of three main stages:

A. Multimodal Encoding & Alignment (Stage I)

• NCS Encoding: Uses pretrained models (e.g., CodeBERT, CodeT5) to extract semantic embeddings.
• CPG Encoding: Uses Relational Graph Convolutional Networks (RGCN) to process the Code Property Graph (capturing Control Flow and Data Dependency).
• Contrastive Alignment: A dual-projection head maps both modalities into a shared metric space using InfoNCE loss to establish semantic correspondence before fusion.

B. Dynamic Fisher Attention (Stage II)

This is the core innovation. Instead of standard attention based on content similarity, the model uses Fisher Information to quantify the sensitivity of feature directions to the classification decision.

• Incremental Fisher Estimation: The model estimates the Fisher Principal Subspace (the directions where feature perturbations most affect the loss) using an efficient online low-rank approximation (Oja's Rule). This avoids the $O(d^3)$ cost of full eigendecomposition.
• Task-Conditioned Query Generation: The attention query vector is enhanced by projecting it onto the Fisher principal subspace. This forces the model to seek structural clues in the CPG that explain high-sensitivity semantic features.
• Complementary Subspace Attention: The CPG features are projected onto the Fisher subspace before attention calculation. This acts as a filter, suppressing noise and redundant features that lie outside the task-sensitive subspace.

C. Adaptive Gating Fusion

Recognizing that not all code samples require structural analysis (e.g., simple buffer overflows vs. complex Use-After-Free), the framework employs a learnable gating mechanism.

• It dynamically calculates a fusion weight ($\alpha$) for each sample based on its semantic complexity.
• If the NCS confidence is high, $\alpha$ decreases to shield the model from graph noise. If NCS confidence is low, $\alpha$ increases to leverage structural evidence.

3. Key Contributions

1. Problem Analysis: Identified and experimentally validated that feature redundancy and modal asymmetry are the primary causes of failure in current multimodal vulnerability detection, rather than a lack of structural data.
2. Theoretical Innovation: Introduced Fisher Information as a metric for task relevance in feature space. Theoretically proved that under isotropic perturbation assumptions, constraining attention to the $k$-dimensional Fisher subspace tightens the output error upper bound from $O(\epsilon)$ to $O(\sqrt{k/d} \cdot \epsilon)$, significantly suppressing noise.
3. Framework Design: Proposed TaCCS-DFA, which combines online Fisher subspace estimation with adaptive gating to achieve selective, task-oriented fusion with minimal computational overhead.
4. Empirical Validation: Demonstrated state-of-the-art performance across four diverse benchmarks (BigVul, PrimeVul, Devign, ReVeal).

4. Experimental Results

The framework was evaluated on four real-world datasets, with a focus on the highly imbalanced BigVul and PrimeVul datasets.

• Performance Gains:
  • On BigVul, TaCCS-DFA achieved an F1 score of 87.80% (using CodeT5-Base), a 6.3 percentage point improvement over the previous best method (Vul-LMGNNs).
  • It significantly improved Recall (from 67.44% to 83.72%) while maintaining high Precision, effectively reducing false negatives.
  • On PrimeVul (3% vulnerability ratio), it achieved the best F1 score (23.68%) among all methods.
• Efficiency:
  • Inference latency increased by only 3.4% compared to standard cross-attention.
  • Training overhead was negligible (+0.9% per batch).
  • GPU memory usage remained stable, showing the method is scalable for DevSecOps pipelines.
• Ablation Studies:
  • Removing Fisher guidance dropped F1 by ~2.1%.
  • Replacing Fisher bases with random orthogonal matrices dropped F1 by ~7.3%, proving the gain comes from task-relevant geometry, not just projection.
  • Randomly rewiring 90% of CPG edges caused a massive 20.9% drop, confirming the model relies on genuine structural information.
• Interpretability: Attention visualizations showed that Fisher-guided attention correctly focused on the "allocation-free-use" chain in Use-After-Free (UAF) vulnerabilities, whereas standard attention diffused over irrelevant lines.

5. Significance

• Practical Impact: By significantly reducing false negatives without sacrificing precision, TaCCS-DFA offers a viable solution for DevSecOps pipelines where missing a vulnerability is far costlier than a false positive.
• Theoretical Shift: The paper shifts the application of Fisher Information from the parameter space (optimization/regularization) to the feature space (fusion/selection), providing a geometric justification for noise suppression in multimodal learning.
• Engineering Value: The method achieves a superior balance between detection accuracy and computational cost, making it suitable for integration into automated security auditing tools for large-scale codebases.