Attesting LLM Pipelines: Enforcing Verifiable Training and Release Claims

1. Problem Statement

Modern Large Language Model (LLM) systems rely heavily on third-party artifacts (pre-trained weights, adapters, datasets, dependencies) assembled via automated pipelines. This "AI supply chain" introduces significant security risks that traditional software supply chain defenses fail to fully address:

• Unverifiable Provenance: Training and release claims (e.g., data lineage, build environment, code history) are rarely cryptographically bound to the resulting artifacts. This leads to ambiguity regarding the origin and integrity of models.
• Malicious Artifacts & Deserialization: Model hubs often blend weights with executable code (custom loaders, converters). Unsafe deserialization mechanisms (e.g., Python pickle) allow attackers to execute arbitrary code during model loading.
• Backdoors & Behavior-Level Threats: Models may contain subtle backdoors activated by rare triggers, which static analysis alone cannot detect.
• Inconsistent Enforcement: Security policies vary across teams and stages, with no unified mechanism to enforce verifiable claims before an artifact enters a trusted environment (training, fine-tuning, or deployment).

2. Methodology

The authors propose an attestation-aware promotion gate that acts as a mandatory admission control point. This gate enforces a strict "claims-to-controls" mapping before allowing artifacts into trusted environments.

A. Claim Taxonomy

The system defines two categories of verifiable claims, bound to the artifact's content digest:

1. Training Claims: Describe the production process.
   • Data Lineage: Dataset IDs, versions, and sampling policies.
   • Code Lineage: Training scripts, configs, and commits.
   • Environment Snapshot: Lockfiles, container digests, and library versions.
   • Hyperparameters & Compute: Optimizers, seeds, and hardware context.
   • Run Metadata: Timestamps and log checksums.
2. Release Claims: Constrain safe consumption.
   • Identity & Format: Artifact digests and safe tensor formats (e.g., safetensors), explicitly disallowing unsafe formats like pickle.
   • Embedded Code Declaration: Explicit flags for custom loaders or converters.
   • Static Scan Results: Summaries of malware and package scans.
   • Deployment Requirements: Policy contracts for permissions (network egress, filesystem access).

B. The Promotion Gate Architecture

The gate operates in four stages:

1. Ingestion: Accepts the artifact and its associated claim bundles (formatted as in-toto attestations signed via Sigstore).
2. Validation:
   • Verifies cryptographic signatures and provenance.
   • Enforces format policies (e.g., rejecting unsafe deserialization).
   • Runs static scanning (using tools like ModelScan and Trivy).
3. Dynamic Evidence (Optional): For high-risk artifacts, the gate can ingest standardized runtime signals (e.g., network egress, process spawning) via plug-ins from existing security tooling.
4. Decision & Enforcement:
   • Allow: If all claims are verified and policies met.
   • Quarantine: If evidence is insufficient or claims are missing (triggering human-in-the-loop triage).
   • Block: If malicious indicators are found.
   • Output: An auditable, machine-readable record of the decision and remediation guidance.

C. Deployment Constraints

Even if an artifact passes the gate, it is deployed under secure-by-default constraints:

• Least-privilege permissions.
• Controlled network egress.
• Restricted filesystem access.
• Centralized logging for post-incident forensics.

3. Key Contributions

1. Attestation-Aware Promotion Gate: A novel architectural pattern that enforces verifiable training and release claims before LLM artifacts enter trusted environments, bridging the gap between software supply chain security and AI-specific risks.
2. Comprehensive Threat Model & Mapping: A detailed mapping of LLM supply chain threats (compromised dependencies, malicious hubs, provenance forgery, backdoors) to specific claims and controls. This includes a taxonomy of training and release claims.
3. Evaluation Blueprint: A proposed framework for evaluating the system across three dimensions:
   • Threat Coverage: Testing against dependency confusion, malicious hub artifacts (MALHUG corpus), and backdoored models.
   • Operational Overhead: Measuring latency and throughput impacts.
   • Triage Usability: Assessing the quality of audit records and the burden on human operators.
4. Practical Implementation Strategy: The paper outlines how to integrate existing tools (Sigstore, in-toto, CycloneDX MLBOM, ModelScan) into a unified pipeline without requiring a complete rewrite of existing infrastructure.

4. Results & Evaluation Plan

Note: As this is a position paper outlining a blueprint, it does not present final empirical results but defines the methodology for obtaining them.

• Case Studies: The authors plan to evaluate the gate against:
  • Dependency confusion attacks (simulating PyTorch registry compromises).
  • Malicious model hub artifacts (using the MALHUG corpus).
  • Backdoored checkpoints (testing quarantine efficacy under insufficient provenance).
• Metrics: Success will be measured by the fraction of threat scenarios mitigated, the latency introduced per artifact, and the reduction in false positives/negatives during human triage.
• Walkthrough Example: The paper demonstrates a scenario where a malicious artifact with unsafe deserialization is blocked or quarantined due to missing signing identity and unsafe format, illustrating how the gate forces remediation (e.g., repackaging into safetensors).

5. Significance

This work addresses a critical gap in AI security by shifting from trust-based to verification-based supply chain management for LLMs.

• Standardization: It proposes a unified standard (using in-toto and CycloneDX) for binding AI-specific metadata (data lineage, hyperparameters) to artifacts, moving beyond simple software bills of materials (SBOMs).
• Risk Reduction: By enforcing "safe-by-default" deployment and cryptographic verification, it significantly reduces the attack surface for supply chain attacks, including code injection and model poisoning.
• Operational Feasibility: The design acknowledges the limitations of current ecosystems (e.g., lack of universal signed provenance) by offering a tiered approach: strict static checks for all, with optional dynamic analysis for high-risk cases, ensuring the system is practical for immediate adoption.
• Future Direction: It sets the stage for a full empirical study to validate the effectiveness of these controls against adaptive adversaries and to refine risk-based policy tuning.