Protecting Deep Neural Network Intellectual Property with Chaos-Based White-Box Watermarking

1. Problem Statement

The rapid proliferation of Deep Neural Networks (DNNs) has created a critical need for Intellectual Property (IP) protection. Trained models represent significant investments in data, computation, and expertise, yet they are easily copied, redistributed, or repurposed without detection.

• The Challenge: Traditional software licensing is insufficient for DNNs, which are often distributed as parameter files. Existing watermarking techniques face several limitations:
  • Fragility: Many methods are easily broken by fine-tuning, pruning, or quantization.
  • Static Signals: Handcrafted or static watermark signals can be filtered out via parameter smoothing or adversarial training.
  • Performance Degradation: Some methods require modifying the training pipeline or retraining from scratch, which is not feasible for third-party or legacy models.
  • Black-box Limitations: While black-box methods exist, they are often less robust and vulnerable to trigger inversion.

2. Methodology

The authors propose a White-Box Watermarking Framework that embeds ownership information directly into the internal weights of a pre-trained DNN using chaotic sequences and verifies ownership using evolutionary algorithms. The framework consists of two main phases:

A. Watermark Generation and Embedding

1. Chaotic Sequence Generation:
   • The watermark is generated using a Logistic Map, a deterministic chaotic function defined as: $x_{n+1} = r \cdot x_n \cdot (1 - x_n)$.
   • Secret Key: The generation relies on three parameters: the logistic constant ($r$), the initial value ($x_0$), and a scaling factor ($\epsilon$). Due to the sensitivity of chaotic systems to initial conditions, these parameters act as a unique, unpredictable signature.
   • The sequence length matches the number of weights in a selected intermediate layer.
2. Embedding Process:
   • The chaotic sequence is injected into the weights of a chosen intermediate layer of a pre-trained model.
   • Modification: Each weight $w_i$ is updated as $w'_i = w_i + \epsilon \cdot c_i$, where $c_i$ is the chaotic value.
   • Fine-Tuning: To minimize accuracy loss, the watermarked model undergoes a brief fine-tuning step on the original training data with a reduced learning rate (1/10th of the original). This restores performance while preserving the embedded signal.

B. Watermark Verification

Verification assumes White-Box access (access to model weights).

1. Signal Extraction: The verifier computes the difference between the suspect model's weights ($W_{ft}$) and a reference watermarked model ($W_{wm}$): $\Delta W = W_{ft} - W_{wm}$. This isolates the embedded watermark signal.
2. Genetic Algorithm (GA) Optimization:
   • Since the original parameters ($r, x_0, \epsilon$) are unknown to the verifier, a Genetic Algorithm is used to search for the optimal parameters that regenerate a chaotic sequence matching the extracted signal.
   • Fitness Function: The GA minimizes a weighted combination of Mean Squared Error (MSE) and Correlation Distance between the extracted signal and the regenerated sequence.
   • Evolutionary Operations: The algorithm uses tournament selection, blend crossover, and Gaussian mutation to evolve the population of candidate parameters toward the original values.
3. Auxiliary Detection:
   • Weight Density Analysis: Visualizing weight distributions to distinguish watermarked models from random or fine-tuned models.
   • Activation-Based Classification: A lightweight Logistic Regression classifier is trained on the internal activation patterns of the DNN to distinguish between original, watermarked, and tampered models.

3. Key Contributions

• Chaotic Sequence Embedding: Introduces a novel method using logistic map-based chaotic sequences, offering high unpredictability and resistance to removal compared to static signals.
• Post-Training Compatibility: The method does not require retraining the model from scratch or modifying the original training pipeline; it operates on pre-trained models with a lightweight fine-tuning step.
• GA-Based Verification: Proposes a robust verification mechanism using Genetic Algorithms to recover the secret chaotic parameters, ensuring reliability even after model perturbations.
• Multi-Layered Verification Strategy: Combines quantitative parameter recovery with qualitative visual analysis (weight density) and machine learning-based classification (logistic regression on activations) for enhanced reliability.
• High Fidelity: Demonstrates that the watermarking process introduces negligible loss in predictive accuracy.

4. Experimental Results

The approach was evaluated on MNIST and CIFAR-10 datasets using Convolutional Neural Networks (CNNs).

• Fidelity (Accuracy):
  • MNIST: The watermarked model showed a marginal accuracy improvement (0.9945 $\to$ 0.9957), suggesting a potential regularizing effect.
  • CIFAR-10: A minor accuracy drop was observed (0.7326 $\to$ 0.7279), but performance remained within acceptable limits.
• Robustness (Fine-Tuning Attacks):
  • The model was subjected to fine-tuning attacks (re-training on the original data with a lower learning rate).
  • Recovery: The Genetic Algorithm successfully recovered the original parameters ($r, x_0, \epsilon$) with high precision. For example, on MNIST, the difference between actual and recovered $r$ was only 0.011, whereas a random model yielded a difference of 0.25.
  • False Positives: The system correctly rejected models watermarked with unrelated parameters, demonstrating resistance to false ownership claims.
• Auxiliary Detection Performance:
  • The Logistic Regression classifier achieved near-perfect accuracy in distinguishing model types:
    • MNIST: 1 error out of 7,920 samples.
    • CIFAR-10: 1 error out of 29,049 samples.
• Convergence: The GA fitness curves showed stable convergence to optimal solutions for watermarked models, while failing to converge to low error values for non-watermarked models.

5. Significance and Limitations

Significance:

• IP Protection: Provides a practical solution for asserting ownership of deployed models without requiring access to training data.
• Resilience: The use of chaotic sequences and evolutionary search makes the watermark difficult to forge or remove via standard fine-tuning.
• Flexibility: Applicable to various stages of the model lifecycle (post-training) and compatible with existing architectures.

Limitations:

• White-Box Requirement: The method requires access to internal weights, limiting its use in black-box scenarios (e.g., MLaaS APIs).
• Computational Cost: The GA-based verification can be computationally expensive for very large models or long sequences.
• Architecture Specificity: Experiments were limited to CNNs; applicability to Transformers or RNNs is unproven.
• Payload Capacity: Currently embeds a continuous chaotic sequence rather than multi-bit identifiers (e.g., specific owner IDs or metadata).
• Attack Scope: Robustness was primarily tested against fine-tuning; robustness against pruning, quantization, or model extraction needs further validation.

Conclusion

The paper presents a robust, white-box watermarking framework that effectively balances IP protection with model utility. By leveraging the unpredictability of chaotic systems and the search capabilities of Genetic Algorithms, the authors demonstrate a method that is resilient to fine-tuning attacks and maintains high model fidelity, offering a viable tool for securing deep learning assets in real-world scenarios.