SoK: From Silicon to Netlist and Beyond $-$ Two Decades of Hardware Reverse Engineering Research

1. Problem Statement

Hardware Reverse Engineering (HRE) is a foundational capability for ensuring the security of modern computing systems, enabling critical tasks such as design verification, supply-chain assurance (detecting hardware Trojans and counterfeits), and vulnerability discovery. However, the field has evolved over two decades into a fragmented discipline, split across three distinct sub-domains:

1. Integrated Circuit (IC) Reverse Engineering: Recovering netlists from physical silicon.
2. FPGA Reverse Engineering: Extracting netlists from proprietary bitstreams.
3. Netlist Reverse Engineering: Analyzing gate-level netlists to recover high-level functionality.

This fragmentation, combined with a lack of standardized benchmarks, shared artifacts, and legal clarity, has hindered the assessment of the state-of-the-art, prevented the identification of shared challenges, and stifled cumulative research progress. The authors argue that the field suffers from a "reproducibility crisis," where technical advances are difficult to verify or build upon due to opaque implementations and missing data.

2. Methodology

The authors conducted a Systematization of Knowledge (SoK) based on a rigorous analysis of 187 peer-reviewed publications spanning from 1985 to 2025. Their methodology followed a four-stage pipeline:

• Step A (Literature Search): Identified 571 papers via database searches (DBLP, Semantic Scholar, Google Scholar) and expanded the corpus to 973 papers through iterative backward and forward citation mining until saturation.
• Step B (Selection & Organization): Filtered papers for relevance, excluding non-peer-reviewed works, tutorials, and papers focused solely on hardware design or side-channel attacks. The final corpus was categorized into IC (87 papers), FPGA (23 papers), and Netlist (77 papers).
• Step C (Distillation): Three research teams analyzed the papers to extract technical approaches (RQ1) and identify open challenges (RQ2) specific to each domain.
• Step D (Artifact Evaluation): The authors performed a retrospective evaluation of all 30 distinct artifacts (tools, images, datasets) claimed in 31 of the papers. They assessed them based on three criteria:
  • Availability: Is the artifact accessible?
  • Functionality: Is it documented, complete, and executable?
  • Reproducibility: Can the key results of the paper be reproduced using the artifact?

3. Key Contributions & Technical Findings

A. Technical Landscape (RQ1)

The paper synthesizes the end-to-end workflows for each domain:

• IC Reverse Engineering: A destructive process involving Sample Preparation (depackaging, delayering via wet/dry etching or ion milling), Imaging (Scanning Electron Microscopy - SEM), Stitching/Stacking (reconstructing 3D layers), Semantic Segmentation (identifying transistors/wires via classical CV or Deep Learning), and Netlist Extraction.
• FPGA Reverse Engineering: A non-destructive process involving Bitstream Format RE (understanding proprietary file structures), Extraction (dumping memory), Decryption (bypassing protections), and Conversion (mapping bits to logic elements and routing).
• Netlist Reverse Engineering: A logical analysis process involving Partitioning (grouping gates into functional cores), Module Identification (matching subcircuits to templates or using Graph Neural Networks), Algorithmic Recovery (reconstructing FSMs and datapaths), and Sensemaking (interpreting the algorithm).

B. Identified Challenges (RQ2)

The authors identified systemic barriers impeding progress:

• Technology Gap: Academic HRE capabilities lag significantly behind industry. While industry has moved to GAAFET nodes (2nm/3nm), academic research predominantly focuses on older planar CMOS nodes (28nm–180nm).
• Error Propagation: In IC RE, errors in sample preparation or imaging propagate through segmentation, making perfect netlist recovery nearly impossible without multiple samples.
• Tooling & Automation: FPGA RE relies heavily on deprecated toolchains (e.g., Xilinx ISE) and lacks support for modern tools (Vivado). Netlist RE suffers from a lack of end-to-end automation; most tools are isolated and non-interoperable.
• Validation Issues: There is a lack of standardized ground truth and metrics. Many papers use synthetic data or minimal benchmarks (e.g., a single adder) that do not reflect real-world complexity.

C. Artifact Evaluation Results (RQ3)

The evaluation of 30 artifacts revealed a critical reproducibility crisis:

• Availability: Only 24 of 30 artifacts were accessible; 5 were lost or unavailable.
• Functionality: Of the 20 executable tools, only 6 ran without issues. Most required significant manual debugging due to poor documentation, missing dependencies, or hardcoded paths.
• Reproducibility: Only 7 papers (4% of the total corpus) allowed the authors to reproduce at least some of the key results. The primary failure points were missing evaluation materials (datasets, netlists) or tools required to compute metrics.

4. Recommendations & Opportunities

The authors propose three cross-cutting opportunities to transform HRE into a rigorous, scalable discipline:

1. Value Reproducibility and Reusability:

   • Academia: Venues should mandate artifact evaluation (similar to USENIX Security) and reward engineering contributions (tools, datasets) as first-class research outputs.
   • Industry: Contribute engineering time to maintain open-source HRE tooling.
   • Researchers: Publish pipelines in containers/VMs with pinned dependencies and modular interfaces.

2. Ensure Rigorous Comparability:

   • Benchmarks: Develop community-driven, legally cleared benchmarks with explicit ground truth (e.g., IC images paired with GDSII, FPGA bitstreams with RTL).
   • Metrics: Adopt domain-adapted metrics (e.g., Electrically Significant Differences) rather than generic pixel-accuracy scores.
   • Living Infrastructure: Treat benchmarks as evolving infrastructure, similar to ML benchmarks (e.g., ImageNet), to keep pace with hardware evolution.

3. Provide Legal Clearance and Clarity:

   • Regulators: Reassess export controls and dual-use regulations that disproportionately hinder public HRE research.
   • Institutions: Provide legal guidance and support for researchers to navigate IP and export restrictions.
   • Industry: Offer research-friendly licensing terms for benchmark use to enable the creation of realistic, legally cleared datasets.

5. Significance

This paper serves as a definitive reference for the HRE community, bridging the gap between hardware design, manufacturing, and security. Its primary significance lies in:

• Exposing the Reproducibility Crisis: By quantifying that only 4% of HRE research is reproducible, it highlights a systemic failure that threatens the validity of the field's security claims.
• Unifying Fragmented Domains: It provides a holistic view of IC, FPGA, and Netlist RE, revealing shared challenges (e.g., error propagation, lack of ground truth) that can be addressed collectively.
• Roadmap for Future Research: It shifts the focus from purely algorithmic novelty to infrastructure building (tools, benchmarks, legal frameworks), arguing that the field cannot mature without coordinated action from academia, industry, and government to enable open, reproducible, and scalable research.

In conclusion, the authors argue that for HRE to effectively secure the hardware supply chain and verify trust, it must evolve from a collection of isolated, opaque experiments into a coordinated, reproducible scientific discipline.