Technical Summary: Semantic Consensus Decoding for Verilog Code Generation

1. Problem Statement

The increasing adoption of Large Language Models (LLMs) for generating Verilog code in hardware design introduces critical security vulnerabilities, specifically backdoor attacks. In these attacks, adversaries poison training data to inject malicious triggers that cause the model to generate hardware trojans (e.g., covert registers for side-channel leakage) when specific trigger patterns appear in the input specification.

The stakes for hardware are uniquely high compared to software:

• Irreversibility: Once a backdoored chip is fabricated, the vulnerability becomes a permanent hardware trojan. Remediation requires costly re-fabrication or is impossible for deployed systems.
• Stealthiness: Advanced triggers (e.g., semantic modifiers like "security" or stylistic sentences) blend naturally into design specifications, evading traditional detection methods.
• Limitations of Existing Defenses:
  • Proactive defenses (data sanitization, retraining) require access to training data, which third-party LLM users typically lack.
  • Passive defenses (input filtering, paraphrasing, output verification) struggle against semantically stealthy triggers. Filtering often fails to detect natural language triggers, paraphrasing can corrupt functional requirements, and verification only detects threats after malicious code is generated.

2. Core Hypothesis and Insight

The authors propose a structural bias in backdoor trigger design driven by the dual requirements of effectiveness (reliably triggering malicious behavior) and stealthiness (preserving model utility on clean inputs).

Hypothesis 1 (Trigger Locality Bias): Attackers are strongly biased toward embedding triggers in non-functional requirements (e.g., style modifiers, quality descriptors, documentation) rather than functional requirements (e.g., bit-width, timing, reset type, I/O specifications).

• Reasoning: Modifying functional requirements risks detection via testbench failures (behavioral discrepancies). In contrast, non-functional modifiers provide semantic space for triggers without altering the hardware's functional correctness.
• Implication: If functional requirements can be isolated and prioritized, the trigger-laden non-functional requirements can be naturally suppressed without compromising the intended hardware behavior.

3. Methodology: Semantic Consensus Decoding (SCD)

SCD is an inference-time passive defense that requires no model retraining or access to clean training data. It operates in two stages:

A. Functional Requirement Extraction

A dedicated extractor model ($E_\phi$) is trained to decompose an input specification $x$ into:

• $s$ (Functional Requirements): Essential specifications determining hardware behavior (e.g., "8-bit," "synchronous").
• $e$ (Non-functional Requirements): Preferences that do not affect verification (e.g., "clean code," "security").

The extractor is trained on a dataset of 9,600 high-quality $\langle$requirement, Verilog code$\rangle$ pairs, constructed via seed annotation by professional engineers, RAG-based generation, and LLM-as-Judge filtering. It is designed to be trigger-agnostic, focusing on domain knowledge rather than specific trigger patterns.

B. Semantic Consensus Decoding

Instead of rigidly discarding non-functional content (which degrades generation quality), SCD adaptively fuses the model's output distributions:

1. Dual Logits Generation: At each decoding step, the model generates logits based on:
   • $\ell_{full}$: The full user specification (containing potential triggers).
   • $\ell_{key}$: The extracted functional requirements only (trigger-free).
2. Divergence Measurement: The system computes the Root Mean Square (RMS) distance ($D$) between $\ell_{full}$ and $\ell_{key}$.
   • Clean Inputs: $D \approx 0$ (distributions align; non-functional tokens affect only style).
   • Triggered Inputs: $D$ is large (distributions diverge; triggers drive the generation).
3. Adaptive Fusion: The final logits ($\ell_{SCD}$) are computed as:
   $$\ell_{SCD} = \ell_{key} + w(D) \cdot (\ell_{full} - \ell_{key})$$
   Where the weight $w(D) = \exp(-\beta \cdot D)$.
   • If $D$ is small (clean), $w(D) \approx 1$, preserving full generation quality.
   • If $D$ is large (triggered), $w(D) \ll 1$, suppressing the trigger-induced shift and falling back to the functional consensus ($\ell_{key}$).

Theoretical analysis (Theorem 1) demonstrates that this mechanism imposes an upper bound on the attacker's effectiveness, as increasing trigger strength simultaneously increases divergence, which exponentially suppresses the trigger's influence.

4. Experimental Results

The authors evaluated SCD on three code LLMs (CodeLlama-7B, DeepSeek-Coder-7B, Qwen2.5-Coder-7B) across two Verilog benchmarks (VerilogEval-v2, ResBench) against three representative attacks (BadPre, InSent, RTL-Breaker).

• Defense Effectiveness (ASR):

  • Without defense, Attack Success Rates (ASR) averaged 89%.
  • SCD reduced the average ASR to 2.16% (VerilogEval) and 1.39% (ResBench).
  • DeepSeek-Coder achieved 0.00% ASR across all attacks and benchmarks under SCD.
  • SCD outperformed existing baselines (ONION, Back-Translation, Paraphrasing) by at least 10 percentage points.

• Generation Quality (Pass@1):

  • Unlike filtering methods that degrade quality, SCD preserved or improved Pass@1.
  • On VerilogEval, SCD improved Pass@1 by up to 8.90 percentage points compared to the benign baseline for CodeLlama.
  • This confirms that extracting functional requirements often yields cleaner, more focused specifications that benefit code generation.

• Robustness:

  • Poisoning Rate: SCD maintained low ASR (<5%) even at high poisoning rates (up to 10%).
  • Model Scale: SCD remained effective across model sizes from 0.5B to 14B parameters.
  • Hyperparameters: A default $\beta = 1.5$ provided an optimal trade-off between security and utility.

5. Key Contributions

1. Hypothesis Validation: The paper provides theoretical analysis and empirical evidence supporting the hypothesis that attackers are biased toward embedding triggers in non-functional requirements due to practical constraints (verification risk, rarity-frequency paradox).
2. Novel Defense Mechanism: The proposal of Semantic Consensus Decoding (SCD), an inference-time defense that adaptively mitigates backdoors by aligning generation with functional requirements, requiring no model retraining.
3. Comprehensive Evaluation: Extensive experiments demonstrating SCD's ability to reduce ASR from ~89% to <3% while maintaining or improving code generation quality across multiple models and benchmarks.
4. Open Source: Release of source code, trained models, and datasets to facilitate replication.

6. Significance and Claims

The paper claims that SCD addresses a critical gap in hardware security by providing a practical, inference-time defense for third-party LLM users who cannot access training data. By leveraging the structural bias of backdoor attacks, SCD effectively neutralizes stealthy triggers that evade traditional detection.

The authors position SCD as a first line of defense specifically effective against the predominant class of backdoor attacks exploiting non-functional requirements. They acknowledge that adaptive attacks embedding triggers within functional requirements remain a challenge, suggesting a "defense-in-depth" strategy where SCD is combined with testbench validation and formal verification. The work highlights that security in hardware design must prioritize irreversibility; thus, preventing the generation of trojans at the inference stage is more viable than attempting to patch vulnerabilities post-fabrication.