Why Neural Structural Obfuscation Can't Kill White-Box Watermarks for Good!

1. Problem Statement

The paper addresses a critical vulnerability in White-Box Watermarking for Deep Neural Networks (DNNs).

• The Threat (NSO): In USENIX Security '23, Yan et al. introduced Neural Structural Obfuscation (NSO), a family of "zero-cost" attacks. NSO injects dummy neurons (channels) into a model while strictly preserving its functional utility (input-output behavior). By using techniques like zero-injection, clique cancellation, and split preservation, NSO scrambles the internal channel indices and layout.
• The Consequence: Most existing white-box watermark verifiers rely on index-ordered assumptions (i.e., they look for watermarks at specific, fixed channel indices). NSO disrupts these indices, causing verification to fail even though the watermark signal remains functionally intact.
• The Gap: Previous defenses were largely limited to simple, sequential architectures and failed to handle modern graph-structured networks (e.g., ResNet, DenseNet) where structural edits propagate through residual connections, fan-outs, and concatenations. The prevailing belief was that NSO could permanently remove watermarks at zero cost.

2. Methodology: The CANON Framework

The authors propose CANON, a recovery framework that redefines NSO not as an irreversible destruction, but as a graph-consistent producer-consumer threat model.

Core Insight

To preserve functional equivalence in a graph-structured network, any structural edit (channel injection) at a "producer" node must induce a consistent, compatible rewrite across all downstream "consumer" nodes (e.g., add, cat, fan-out). This creates global constraints on signal propagation that are exploitable for recovery.

Key Components of CANON

1. Graph-Consistent Threat Model:

   • Instead of viewing NSO as layer-local edits, CANON models it as a ChannelTransform ($M$) applied to producer edges.
   • It enforces that for any downstream consumer (like a residual add or concatenation), the layout changes must be synchronized. For example, in a residual add, both branches must share the exact same channel layout; in cat, the transforms must compose into a block-diagonal matrix.

2. Redundancy Detection (Probe-Based):

   • CANON uses a small set of probe inputs to collect activation signatures.
   • Signature Hashing: It computes binary activity patterns (state bits) for each channel across probes. Channels with identical signatures are grouped into coarse buckets.
   • Proportionality Refinement: Within buckets, it estimates scaling factors ($\alpha$) to distinguish between true redundancy (dummy channels) and benign structural components. It checks if the weighted sum of a cluster's contributions cancels out (for nso_clique) or reconstructs the original (for nso_split).

3. Global Canonicalization (The Recovery):

   • Fan-out Aware: Once a redundant cluster is identified on a producer edge, CANON synthesizes a sparse ChannelTransform ($M_e$) to compact the output.
   • Consumer Rewriting: Crucially, it does not just prune the layer. It rewrites all downstream linear consumers (Conv/Linear weights) using the rule $W_{new} = W_{old} \times M_e$. This ensures functional equivalence is preserved.
   • Synchronization: It propagates these layout corrections through the entire graph, enforcing consistency at add nodes (synchronizing branches) and cat nodes (block-diagonal composition).

4. Two-Tier Verification:

   • Tier 1 (Reference-Equivalence Certificate): If a clean pre-attack reference model is available, CANON verifies that the recovered weights match the reference up to allowed symmetries (permutation and positive scaling).
   • Tier 2 (Similarity Fallback): If no reference exists, it uses an attack-aware similarity metric to ensure the recovered model yields high watermark scores.

3. Key Contributions

• Reformulation of NSO: The paper generalizes NSO into a graph-consistent producer-consumer model, exposing the signal-consistency constraints that are necessary for the attack to work but also serve as the key to defeating it.
• Constructive Recovery Proof: Contrary to the belief that NSO is a "zero-cost removal," the authors prove constructively that NSO induces structured, globally constrained redundancy that can be identified and reversed without knowing the attacker's specific construction details.
• CANON Framework: The first end-to-end framework that defeats NSO by graph-consistent canonicalization. It recovers a compact, verifiable parameterization without requiring the original clean model.
• Empirical Invalidation: The authors demonstrate that NSO fails as a permanent removal strategy. Under strong, compositional attacks, CANON restores watermark verifiability with 100% success and zero accuracy loss.

4. Experimental Results

The authors evaluated CANON on four modern architectures (ResNet-18, EfficientNet, InceptionV3, DenseNet) and nine diverse white-box watermarking schemes (weight-based, activation-based, and passport-based).

• Recovery Success: Across all models and attack variants (including the strongest mix-opseq composed attacks), CANON achieved 100% recovery of watermark verifiability.
• Utility Preservation: The recovery process introduced zero functional drift. The accuracy of the recovered model ($\hat{\theta}$) was identical to the attacked model ($\theta'$) and the clean model ($\theta$).
• Robustness to Attack Strength: The results held true even at high injection ratios ($\rho = 0.5$) and under complex, graph-aware attack compositions.
• False Positives: When applied to clean (non-attacked) models, CANON made zero structural changes, preserving accuracy and watermark similarity perfectly.
• Efficiency: Recovery runtime is dominated by the complexity of the NSO attack (specifically compositional attacks) but remains practical (typically under 1 minute), making it a viable one-time verification step.

5. Significance

This paper fundamentally shifts the landscape of neural network watermarking security:

1. Demystifies NSO: It proves that "zero-cost" structural obfuscation is not a silver bullet for removing watermarks. The very constraints required to maintain functional equivalence in graph-structured networks make the attack reversible.
2. Restores Viability: It re-establishes the practicality of white-box watermarks for ownership verification, even in the face of sophisticated structural attacks.
3. New Defense Paradigm: It moves the field from "robustness against value perturbations" (fine-tuning, pruning) to "recovery from structural obfuscation," introducing the concept of graph-consistent canonicalization as a standard defense mechanism.

In conclusion, the paper demonstrates that while NSO can temporarily break index-based verification, it cannot destroy the underlying watermark signal. By leveraging the mathematical necessity of graph consistency, defenders can fully recover the original watermark-verifiable state.