Compliant But Unsatisfactory: The Gap Between Auditing Standards and Practices for Probabilistic Genotyping Software

1. Problem Statement

The paper addresses a critical failure in AI governance: audit standards can be poorly designed such that they allow systems to be "compliant" while remaining technically inadequate or dangerous. Specifically, the authors investigate ASB 018, a standard developed by the American Academy of Forensic Sciences Standards Board (ASB) for auditing Probabilistic Genotyping Software (PGS).

PGS is widely used in the U.S. criminal legal system to analyze complex DNA mixtures and generate Likelihood Ratios (LRs) as evidence. While ASB 018 was intended to ensure these systems are reliable and to establish boundaries for their use, the authors hypothesize that the standard's design allows forensic laboratories to produce audit reports that satisfy the standard's formal requirements (letter of the law) but fail to achieve its intended safety goals (spirit of the law). This creates a risk of "audit washing," where non-rigorous audits lend false credibility to potentially flawed forensic evidence presented in court.

2. Methodology

The authors employed a qualitative case study approach combining document analysis of the standard and empirical audit reports.

• Data Collection:
  • The Standard: ASB 018 (First Edition, 2020) and its accompanying factsheet.
  • Audit Reports: Five publicly available internal validation studies (audits) for the STRmix PGS (the leading PGS in the U.S.), spanning different labs (NYC OCME, Colorado CBI, Palm Beach PBSO, Maryland MSP), software versions (v2.4 to v2.9.1), and years (2016–2024).
• Analytical Framework:
  • The study utilized Ojewale et al.'s [59] five-stage audit framework: Audit Scope, Standards Identification, Performance Analysis, Post-Audit Judgment, and Audit Communication.
  • Compliance Assessment: The authors first assessed whether the five reports could be interpreted as compliant with ASB 018. They adopted a "permissive interpretation" of the standard's language (e.g., interpreting vague terms like "consider" or "address" in the most lenient way possible) to determine the maximum gap between the standard's potential and its actual enforcement.
  • Gap Analysis: They qualitatively coded the standard and the reports to identify:
    1. Envisioned Goals: What the standard intends to happen (derived from the text and factsheet).
    2. Realized Practices: What the labs actually did.
    3. Design Enablers: Specific features of the standard's language (ambiguity, undefined terms, structural separation) that allowed the gap to exist.

3. Key Contributions

• Empirical Evidence of "Compliant but Unsatisfactory" Audits: The paper demonstrates that all five analyzed audits were technically compliant with ASB 018 yet failed to meet the standard's core safety goals across all five audit stages.
• Identification of Design Flaws in Audit Standards: The authors pinpoint specific linguistic and structural mechanisms in ASB 018 that enable these gaps, moving beyond general criticism to specific design recommendations.
• Framework for Evaluating Audit Standards: The study provides a methodological template for evaluating how audit standards can inadvertently legitimize inadequate systems, applicable to other domains like hiring or healthcare.

4. Key Results: The Five Gaps

The authors identified systematic misalignments between ASB 018's goals and the practices it enables:

A. Audit Scope

• Goal: Evaluate the broader sociotechnical system (software + human analyst inputs).
• Practice: Audits tested the software in isolation, often using "ground truth" parameters (e.g., the correct number of contributors) rather than the analyst's estimates. Some audits appeared to be conducted by software developers rather than independent lab analysts.
• Design Enabler: The standard defines the "system" narrowly as software/hardware, separating "user input parameters" as a distinct, optional consideration rather than an integral part of the performance test.

B. Standards Identification (Defining Expectations)

• Goal: Test against predetermined, falsifiable specifications aligned with customer (e.g., prosecutor/defense) needs.
• Practice: Labs rarely defined explicit thresholds for "acceptable" performance. Instead, they used subjective terms like "high," "low," "intuitive," or "expected" to describe results. No audit mentioned customer needs or perceptions.
• Design Enabler: The standard requires labs to "address" accuracy/sensitivity but does not mandate that they define what "acceptable" means or that these definitions be falsifiable.

C. Performance Analysis

• Goal: Measure performance across a representative range of casework samples (including degraded DNA, mixtures of relatives, etc.).
• Practice: Audits tested narrow subsets of samples (e.g., only one-person samples for accuracy, ignoring complex mixtures). Sample sizes were often insufficient, and labs did not justify their sample size choices.
• Design Enabler: The standard lists specific sample characteristics (contributors, ratios, DNA amount) but omits others (degradation, allele sharing) from the mandatory requirements, allowing labs to ignore critical failure modes. The term "sufficient studies" is undefined.

D. Post-Audit Judgment

• Goal: Establish boundaries on software use based on observed failures (e.g., "Do not use this software for X type of sample").
• Practice: Audits rarely labeled failures as "unacceptable." Instead, failures were framed as "non-intuitive" or "not unexpected," with the assumption that a "trained analyst" would catch the error. No specific boundaries on sample types were established.
• Design Enabler: The standard uses the undefined term "limitations" and does not explicitly require labs to specify these limitations or to treat observed errors as grounds for restricting use.

E. Audit Communication

• Goal: Documentation must enable third-party scrutiny and reproducibility.
• Practice: Reports omitted crucial details (exact LR values, specific sample ranges) needed for independent verification. Data was often presented in aggregate or with vague descriptors.
• Design Enabler: The requirement for third-party examination is phrased as a non-mandatory "incumbent upon" (moral obligation) rather than a strict "shall" requirement, and the standard fails to list specific data points that must be disclosed.

5. Significance and Recommendations

Significance:
The paper challenges the assumption that compliance with a standard equates to safety or reliability. In the context of the criminal legal system, where PGS evidence can determine liberty or life, "compliant but unsatisfactory" audits pose a severe risk to due process. The study highlights that designing standards for compatibility with current user practices (flexibility) can actively undermine the tool's effectiveness by legitimizing inadequate workflows.

Recommendations for Designing Audit Standards:

1. Articulate Fine-Grained Outcomes: Standards must clearly define desired outcomes (e.g., "establish boundaries") and map specific requirements to these outcomes, rather than relying on vague goals.
2. Specify Actors and Responsibilities: Clearly define who must do what (e.g., the lab vs. the developer) to prevent ambiguity and ensure independence.
3. Define Minimum Activities and Intermediary Steps: Replace vague verbs like "consider" or "address" with specific, measurable actions (e.g., "define a threshold for X," "test Y number of samples").
4. Co-Design with Adversarial Stakeholders: Engage groups beyond the standard's primary users (e.g., defense attorneys, independent experts) to counterbalance the industry's desire for flexibility and ensure the standard addresses the full spectrum of risks, including those relevant to the accused.

In conclusion, the authors argue that for audit standards to be effective tools for accountability, they must move from flexible guidelines to rigorous, verifiable requirements that explicitly prevent the "compliant but unsatisfactory" loophole.