Gaming the Metric, Not the Harm: Certifying Safety Audits against Strategic Platform Manipulation

Technical Summary: Gaming the Metric, Not the Harm

Problem Statement
Recent regulatory frameworks, specifically the UK Online Safety Act and the EU Digital Services Act, increasingly rely on scalar metrics as evidence of compliance for online safety. This paper identifies a critical security vulnerability: once a metric is published, it becomes an optimization target for the platform. A strategic platform can improve its safety score without reducing actual harm by routing recommendations through semantically equivalent content variants that receive lower scores from the auditor's detector. This constitutes a form of "reward hacking" or Goodhart's law, where the metric is gamed rather than the underlying harm being mitigated. The paper isolates the specific attack surface of within-class representation choice under a frozen, published audit protocol, asking when a safety metric can still certify a genuine upper bound on harmful exposure despite such strategic manipulation.

Methodology and Setup
The authors model the audit as a game between an auditor and a platform.

• Protocol Definition: A published protocol defines a finite set of content variants $\mathcal{V}$ and a transformation family $T_0$. Through a validation rule and threshold $\rho$, the protocol induces a partition of $\mathcal{V}$ into semantic classes $\mathcal{C}$. Variants within a class are considered semantically equivalent for audit purposes.
• Platform Strategy: The platform chooses a probability distribution $x$ over variants to maximize utility subject to an audit budget constraint ($M(x) \leq \tau$).
• Threat Model: The platform can swap variants within the same semantic class (e.g., changing a caption, thumbnail, or paraphrase) to alter the measured score $m(v)$ while keeping the latent harm $h^*(v)$ constant.
• Evaluation: The paper employs a multi-layered verification approach:
  1. Theoretical Proofs: Formal definitions of manipulation invariance and certification.
  2. Exhaustive Enumeration: Checking all mixed strategies on a finite-state grid.
  3. SMT Encoding: Using Z3 and cvc5 to verify properties over the continuous simplex.
  4. Bounded MDP: Modeling the audit as a sequential decision process in PRISM-games.
  5. Synthetic Stress Tests: Using deterministic and random catalogs, including a HateCheck-derived protocol example, to measure the "gaming gap."

Key Contributions

1. Formalization of Metric Robustness: The paper formalizes "manipulation invariance" (the metric should not change if the platform swaps semantically equivalent variants) and "certification" (the metric must upper-bound true harmful exposure) as security properties of audit metrics.
2. Proof of Fragility: It is proven that any metric scoring variants directly is manipulable if two equivalent variants in a harmful class have different scores.
3. The Semantic-Envelope Repair: The authors propose a "semantic-envelope lift," where every variant in a semantic class is assigned the maximum score attained by any admissible variant in that class ($Env(m)(v) = \max_{u \in cl(v)} m(u)$).
   • Theorem: This lift is the unique pointwise minimum among all conservative, classwise-constant repairs. It is the least conservative fix that restores invariance.
4. Certification Theorems:
   • Ideal Case: If classes are harm-pure, the envelope metric provides a global certificate $H(x) \leq (1/\alpha) M_{Env}(x)$, where $\alpha$ is the minimum maximum-score across harmful classes.
   • Imperfect Case: The framework extends to imperfect protocols and annotations by introducing a "disagreement mass" term $\Delta(x)$. The resulting certificate is $H^*(x) \leq (1/\hat{\alpha}) M_{Env}(x) + \bar{\eta}$, where $\bar{\eta}$ bounds the disagreement between latent harm and audited labels.
5. Protocol Sensitivity: The paper demonstrates that tightening the protocol (raising the validation threshold) refines the semantic partition, which weakly decreases the envelope score pointwise, allowing auditors to report a monotone sensitivity band.

Results

• Fragility of Direct Scoring: In synthetic experiments, the "fragile metric" (direct scoring) allows platforms to achieve maximal true harmful exposure (e.g., $H=1.0$) while staying within a low audit budget (e.g., $\tau=0.10$).
• Effectiveness of the Envelope: The semantic-envelope metric forces the platform to substitute toward benign content to meet the same budget. In the deterministic case study, at $\tau=0.10$, the envelope metric reduced true harmful exposure to $0.000$ compared to $1.000$ for the fragile metric.
• Random Catalogs: Across 500 random catalogs, the mean "gaming gap" (difference in true harm between fragile and envelope strategies) was $0.884$. The fragile metric consistently allowed higher utility at the cost of significantly higher harm.
• Verification: All SMT queries and MDP reachability checks confirmed that the envelope metric satisfies manipulation invariance and certification properties, while the fragile metric fails invariance.
• HateCheck Example: A protocol derived from the HateCheck functional test suite confirmed that the attack surface and the efficacy of the envelope repair persist even when classes are grounded in real-world benchmark taxonomies.

Significance and Claims
The paper makes a modest but specific claim: audit metrics should be treated as security objects. The authors do not claim to solve the entire platform-audit problem or to provide a deployable solution for Ofcom or DSA-scale audits immediately. Instead, they argue that:

1. Measurement Hardening: A simple "max-over-class" repair (the semantic envelope) removes a well-defined attack surface (within-class manipulation) without requiring changes to the underlying classifier or protocol renegotiation.
2. Explicit Uncertainty: Annotation and protocol errors should not be hidden assumptions but should appear explicitly as a "disagreement slack" term in the final safety certificate.
3. Utility Agnosticism: The theoretical guarantees hold for all platform strategies, regardless of the specific utility function the platform optimizes; the utility model is only used in experiments to select a specific adversarial best response.

The work positions itself as a foundational step in "measurement hardening," isolating the within-class manipulation surface and providing a formal, verifiable repair that restores invariance and certification. It explicitly excludes broader meta-games like protocol negotiation or user-side manipulation, noting those require separate analysis.