Technical Summary: Hidden Ciphers and Where to Find Them

Problem Statement

Modern software systems rely heavily on cryptography for data protection, authentication, and trust establishment. However, organizations frequently lack a structured, holistic view of the cryptographic assets deployed across their source code, configurations, dependencies, and binary artifacts. This lack of visibility hinders effective security governance and complicates critical tasks such as post-quantum (PQ) migration planning.

Existing solutions often address this problem only partially. Many tools focus narrowly on known misuse patterns or couple detection logic tightly with assessment knowledge, limiting maintainability and extensibility. Furthermore, there is no unified classification for cryptographic assets, leading to fragmented discovery efforts where it remains unclear which assets should be found or how findings should be grouped. Consequently, cryptographic inventorying often relies on ad hoc checks rather than a structured, repeatable process.

Methodology

The authors employ the Design Science Research (DSR) methodology to address the problem of missing cryptographic visibility. This approach involves designing and evaluating a set of interconnected artifacts to solve a concrete real-world problem. The development process is guided by four research questions (RQs) and iterates through relevance, rigor, and design cycles.

The core of the methodology is a classification-driven static analysis approach, consisting of four key artifacts:

1. A Classification of Cryptographic Assets: A taxonomy distinguishing between three discovery categories and two assessment categories.
2. Crypistry: An extensible, scanner-independent rule repository.
3. Crypsy: A static scanner that implements the rules to discover and assess assets.
4. Cryben: A synthetic benchmark corpus with known ground truth for evaluation.

Key Technical Components

1. Classification of Cryptographic Assets

The paper proposes a dual-dimensional classification system:

• Discovery Dimensions (Structural):
  • Cryptographic Material: Value-centric assets (e.g., private keys, seeds, tokens) where the security role of the value is paramount.
  • Cryptographic Artifacts: Structure-centric objects (e.g., X.509 certificates, keystores, PKCS#12 files) containing security-relevant metadata.
  • Cryptographic Invocations: Context-centric usage points (e.g., API calls, configuration settings, dependency declarations) that select or configure cryptographic functionality.
• Assessment Dimensions (Evaluative):
  • Weakness: Security-relevant properties grounded in taxonomies like CWE (e.g., weak algorithms, hard-coded keys).
  • Vulnerability: Concrete, known security issues grounded in databases like CVE (e.g., vulnerable library versions, specific implementation flaws).

2. Crypistry (Rule Repository)

Crypistry is a structured repository that separates domain knowledge from scanner logic. It contains two types of rules:

• Discovery Rules: Define how to detect assets. They are categorized by asset type (Material, Artifact, Invocation) and specify scan methods (file parsing, AST analysis, regex matching) and file globs.
• Assessment Rules: Define how to evaluate discovered assets. They map findings to CWE or CVE identifiers and operate on normalized asset components rather than raw files.
  This separation allows the rule set to evolve independently of the scanner, supporting extensibility for new languages, libraries, and security knowledge.

3. Crypsy (Static Scanner)

Crypsy is a rule-based static scanner that processes software "at rest" (source code, configs, manifests). It operates in a deterministic four-phase pipeline:

1. Rule Loading: Loads discovery and assessment rules from Crypistry.
2. Discovery: Collects files, classifies them based on rules, extracts raw findings via format-specific parsers or AST analysis, and normalizes them into canonical CryptoComponent objects.
3. Assessment: Evaluates normalized components against weakness and vulnerability rules to assign severity and mappings.
4. Export: Generates structured output, specifically supporting CycloneDX v1.7 Cryptographic Bill of Materials (CBOM).

Key Contributions

The paper makes four primary contributions:

1. A Unified Classification: A taxonomy distinguishing Crypto-Material, Crypto-Artifacts, and Crypto-Invocations, alongside Weakness and Vulnerability assessment categories.
2. Crypistry: An extensible, scanner-independent rule repository that organizes detection and assessment knowledge, mapping findings to standard taxonomies (CWE, CVE).
3. Crypsy: A static scanner that implements this classification and rule set to produce CBOM-oriented output.
4. Empirical Evaluation: A rigorous assessment using a synthetic benchmark (Cryben) and a real-world infrastructure of ten deployed services.

Evaluation Results

Benchmark Evaluation (Cryben)

The authors evaluated Crypsy against a synthetic benchmark (Cryben) containing 197 ground-truth occurrences across Go, Ruby, and nginx configurations.

• Discovery Performance: Crypsy achieved an F1 score of 0.75 (Precision: 0.87, Recall: 0.66).
  • Material: Near-perfect recall (F1 = 0.94).
  • Invocations (Go): Strong performance (F1 = 0.92).
  • Invocations (Ruby): Lower recall (F1 = 0.51) due to a counting-model mismatch where the scanner groups calls per object while ground truth counted individual method calls.
  • Artifacts: F1 = 0.61, with false negatives attributed to coverage gaps in uncommon PEM label variants.
• Assessment Performance: Crypsy correctly annotated 91% of expected weaknesses and vulnerabilities (Assessment Recall = 0.91, F1 = 0.74).
• Comparison: Crypsy outperformed the existing tool CBOMkit-hyperion on the Go-invocation subset, achieving an F1 of 0.92 compared to CBOMkit's 0.66.

Real-World Evaluation

Crypsy was deployed on a self-operated IT infrastructure comprising ten services (including GitLab, Mastodon, Harbor, and various databases) with 57,610 files.

• Efficiency: The scanner processed all files in under six minutes.
• Discovery: It identified 370 cryptographic assets. All manually identified assets were detected (partial recall of 100% against the manual ground truth).
• Assessment:
  • 91 components received CWE-mapped weakness annotations.
  • 6 components received CVE-linked vulnerability annotations (including Heartbleed, BEAST, and library-specific flaws).
  • 52 components were identified as candidates for post-quantum migration (RSA/EC keys and certificates vulnerable to Shor's algorithm).
• Actionability: While some weakness annotations (e.g., weak hash usage) required manual triage to distinguish between security-critical and non-critical uses, posture-related weaknesses (e.g., disabled TLS validation) were directly actionable.

Significance and Claims

The paper claims that classification-driven static discovery can provide practical cryptographic transparency for governance and post-quantum migration planning. By decoupling detection logic from assessment knowledge via the Crypistry repository, the approach offers a maintainable and extensible foundation for cryptographic inventorying.

The results demonstrate that a structured, rule-based approach can effectively discover and assess cryptographic assets across diverse environments, producing standards-compliant CBOMs. This capability enables organizations to move from ad hoc checks to a repeatable process for understanding their deployed cryptography, a prerequisite for managing security risks and planning migrations to post-quantum standards.

The authors remain modest regarding generalizability, noting that current support is focused on Ruby and Go ecosystems and specific configuration formats. They acknowledge that extending coverage to other major languages (Java, C/C++, Python) and validating against exhaustive ground truths in diverse environments are necessary next steps. The paper does not claim to solve cryptographic agility or migration strategy generation but positions its CBOM output as a critical evidence base for those future processes.