💬 NLP

Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test

This paper proposes a rank-based uniformity test that enables users to accurately and efficiently audit black-box LLM APIs for unauthorized model substitutions, such as quantization or harmful fine-tuning, by verifying behavioral consistency with a local authentic model without revealing detection patterns.

Original authors: Xiaoyuan Zhu, Yaowen Ye, Tianyi Qiu, Hanlin Zhu, Sijun Tan, Ajraf Mannan, Jonathan Michala, Raluca Ada Popa, Willie Neiswanger

Published 2026-03-20

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaoyuan Zhu, Yaowen Ye, Tianyi Qiu, Hanlin Zhu, Sijun Tan, Ajraf Mannan, Jonathan Michala, Raluca Ada Popa, Willie Neiswanger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you order a "Premium Steak" from a high-end restaurant. You expect a juicy, high-quality cut. But secretly, the chef swaps it for a cheaper, frozen patty to save money. You can't see the kitchen, and you can't ask to see the raw ingredients (the model's code). All you get is the final plate (the text response). How do you know if you're getting what you paid for?

This is the exact problem facing users of Large Language Model (LLM) APIs today. Companies like OpenAI, Google, or Anthropic provide access to powerful AI models via the internet. However, because these models are "black boxes," users have no way to verify if the provider is actually using the expensive, high-quality model they promised, or if they've swapped it for a cheaper, lower-quality version (like a "quantized" or "fine-tuned" model) to cut costs or sneak in bad behavior.

The paper "Auditing Black-Box LLM APIs with a Rank-Based Uniformity Test" proposes a clever, invisible way to catch these cheaters. Here is the breakdown in simple terms:

The Problem: The "Magic Box" Deception

Imagine you have a Magic Box (the API) that claims to be a specific, famous chef (the authentic model).

The Catch: You can only ask the box to write a story or solve a math problem. You can't open the box to see the chef inside.
The Risk: A dishonest provider might swap the famous chef with a cheap intern (a smaller, quantized, or hacked model). The intern might write slightly different sentences, make small grammar errors, or choose different words.
The Old Way: Previous methods tried to catch this by asking the box very specific, weird questions (like "Write a poem about a toaster in the style of Shakespeare"). But a smart cheater could recognize these "test questions," realize they are being audited, and temporarily switch back to the real chef just for those questions. It's like a student memorizing the test answers but failing the real exam.

The Solution: The "Rank-Based Uniformity Test" (RUT)

The authors propose a new method called RUT. Instead of asking weird questions, they use a statistical trick that works like a blind taste test.

1. The Setup: The "Reference Chef"

The auditor (the person checking the API) has a copy of the real chef's recipe book running on their own computer. This is the Reference Model. They know exactly how the real chef behaves.

2. The Trick: The "Rank" Game

Here is how the test works, step-by-step:

Step A: The auditor asks the Target API (the suspicious box) a normal question, like "What is the capital of France?"
- Result: The API says "Paris."
Step B: The auditor asks their Reference Chef the exact same question 100 times.
- Result: The Reference Chef says "Paris" 90 times, "Lyon" 5 times, "Marseille" 3 times, and "Nice" 2 times.
Step C: The auditor looks at the Target API's answer ("Paris") and asks: "Where does 'Paris' rank in the Reference Chef's list?"
- Since "Paris" is the most common answer for the real chef, it is at the very top of the list (Rank #1).
Step D: They repeat this for hundreds of different questions.

3. The "Uniformity" Check

If the Target API is truly the same as the Reference Chef, their answers should follow the exact same distribution.

If the Reference Chef says "Paris" 90% of the time, the Target API should also say "Paris" 90% of the time.
If the Target API is a "cheater" (a different model), their answers will be distributed differently. Maybe they say "Paris" 99% of the time, or maybe they say "London" 10% of the time.

The math behind RUT checks if the "ranks" of the answers are uniformly distributed (spread out evenly like a deck of cards).

If they are uniform: The API is honest.
If they are bunched up or weird: The API is lying.

Why is this better? (The Superpowers)

It's Invisible (Stealth):
The auditor uses normal, everyday questions (like "What's the weather?" or "Write a poem"). The cheater can't tell they are being tested because the questions look like normal user traffic. They can't "game the system" by switching chefs because they don't know which questions are tests.
It's Efficient (Cheap):
Previous methods needed to ask the API thousands of questions to get a good answer. RUT only needs one question per prompt from the API, while doing the heavy lifting on the auditor's own computer. It's like checking a single grain of rice to see if the whole bag is fake, rather than eating the whole bag.
It Catches Everything:
The paper tested RUT against many types of cheating:
- Quantization: Swapping a high-res photo for a blurry one (making the model smaller).
- Jailbreaking: Sneaking in hidden instructions to make the model say bad things.
- Fine-tuning: Teaching the model to be rude or follow different rules.
- Full Swaps: Replacing the model entirely with a different one.
In almost every case, RUT caught the cheaters faster and more accurately than previous methods.

The Analogy: The "Coin Flip" Test

Imagine you suspect a casino is using a weighted coin (a fake coin) instead of a fair one.

Old Method: You ask the casino to flip the coin 1,000 times in a row and count the heads. If they see you counting, they might switch to a fair coin just for that moment.
RUT Method: You bring your own fair coin (Reference Model). You flip your fair coin 100 times and write down the results. Then, you ask the casino to flip their coin once for the same "scenario." You check if their single flip matches the pattern of your 100 flips. If their coin is weighted, the pattern of their single flip (when repeated over many different scenarios) will look statistically "off" compared to your fair coin's pattern. You catch them without them ever knowing you were testing them.

Conclusion

This paper gives us a powerful new tool to ensure that when we pay for "Premium AI," we actually get "Premium AI." It's a digital lie detector that doesn't need to see the engine to know if the car is running on the right fuel. By using simple statistics and normal conversation, it exposes dishonest providers who try to cut corners, ensuring the safety and reliability of the AI tools we use every day.

1. Problem Statement

As Large Language Models (LLMs) are increasingly accessed via APIs, users face a "black-box" scenario where they lack visibility into the underlying model implementation. API providers may covertly substitute the claimed model with cheaper or altered variants to reduce costs or introduce malicious behaviors. Common substitution strategies include:

Quantization: Serving 4-bit or 8-bit quantized versions of the model.
Fine-tuning: Applying harmful fine-tuning or jailbreak-enabling system prompts.
Full Substitution: Replacing the model entirely with a different architecture.
Probabilistic Routing: Mixing responses from the claimed model and an alternative model based on prompt detection (e.g., rerouting suspicious queries).

The Challenge: Detecting these substitutions is difficult because users typically lack access to model weights or output logits. Existing detection methods often fail due to:

Low Query Efficiency: Requiring excessive API calls (costly).
Lack of Robustness: Relying on specific query patterns that adversarial providers can detect and bypass (e.g., via prompt caching).
Low Accuracy: Failing to distinguish subtle distributional shifts caused by quantization or minor fine-tuning.

2. Methodology: Rank-Based Uniformity Test (RUT)

The authors propose RUT, an asymmetric two-sample hypothesis test designed to verify if a target API model ( $\pi_{tgt}$ ) is behaviorally identical to a locally deployed reference model ( $\pi_{ref}$ ).

Core Concept

The method relies on the principle that if $\pi_{tgt} = \pi_{ref}$ , the target model's responses should appear as a random sample drawn from the reference model's distribution.

Score Function: For a given prompt $x$ , a scalar score $s = f(y, x)$ is assigned to a response $y$ . The paper empirically selects Log-Rank (the average of the logarithm of token ranks based on the reference model's probabilities) as the most discriminative score function.
Rank Statistic: For each prompt, the auditor queries the target API once ( $y_{tgt}$ $y_{t g t}$ ) and the reference model $m$ $m$ times ( $y_{ref, j}$ $y_{r e f, j}$ ). The rank of the target score within the reference distribution is computed.
- To handle discrete distributions, a randomized quantile residual is applied: $r_{tgt} = F_{ref}(s_{tgt}^-) + U \cdot P(s_{ref} = s_{tgt})$ , where $U \sim \text{Uniform}[0,1]$ .
- Under the null hypothesis ( $H_0$ : models are identical), $r_{tgt}$ follows a Uniform[0, 1] distribution.
Hypothesis Testing: The auditor collects $n$ such rank statistics. A Cramér–von Mises (CvM) test is applied to check if the collection of ranks deviates significantly from a uniform distribution. If the p-value is low, $H_0$ is rejected, indicating model substitution.

Key Design Features

Query Efficiency: Requires only one API call per prompt (plus local reference calls), making it cost-effective.
Stealth: Uses natural, user-like prompts rather than specially crafted "fingerprinting" queries, making it difficult for providers to detect and reroute testing traffic.
Robustness: Effective against probabilistic substitution (where only a fraction of queries are routed to a different model).

3. Key Contributions

Novel Auditing Framework: Introduction of RUT, a method that satisfies accuracy, query efficiency, and robustness simultaneously, addressing the limitations of prior work like Maximum Mean Discrepancy (MMD) and identity prompting.
Empirical Validation: Comprehensive evaluation across diverse threat models:
- Quantization (4-bit/8-bit).
- Jailbreaking (hidden system prompts).
- Supervised Fine-Tuning (SFT) on benign and harmful data.
- Full model replacement.
- Specialized domains (Math and Code).
Real-World Deployment: Benchmarking against live commercial APIs (e.g., Nebius, Novita, Hugging Face Inference), demonstrating the method's practical utility in detecting deviations in real-world settings.

4. Experimental Results

The authors evaluated RUT against MMD (Gao et al., 2025) and a Kolmogorov–Smirnov (KS) baseline using Statistical Power AUC (Area Under the Curve of Power vs. Substitution Rate) as the primary metric.

Quantization Detection: RUT significantly outperformed baselines. For example, in detecting 4-bit quantization of Llama-3.2-3B, RUT achieved an AUC of 0.642, compared to MMD (0.625) and KS (0.474). While 8-bit quantization proved difficult for all methods, RUT remained the most sensitive.
Jailbreak Detection: RUT achieved the highest power in all 6 tested jailbreak scenarios (Dan, Anti-Dan, Evil-Bot), with AUCs consistently above 0.85, outperforming MMD and KS.
SFT Detection: RUT detected behavioral changes within the first epoch of fine-tuning, showing superior sensitivity to early-stage distributional shifts compared to baselines.
Full Model Replacement: RUT consistently achieved the highest AUC across pairwise comparisons of different model families (e.g., Gemma vs. Llama vs. Mistral).
Robustness to Decoding Mismatches: In a case study involving real API providers, RUT correctly identified behavioral equivalence where string-based metrics (like MMD using Hamming distance) failed due to minor formatting differences (e.g., leading whitespace).
Query Domains: The method maintained high effectiveness on specialized datasets like BigCodeBench (coding) and MATH, proving generalizability beyond general conversational data.

5. Significance and Impact

Trust and Safety: RUT provides a practical tool for users and regulators to verify that API providers are delivering the models they claim, preventing performance degradation and security risks (e.g., hidden jailbreaks).
Cost-Effective Auditing: By requiring minimal API queries and avoiding detectable patterns, RUT enables frequent, large-scale auditing without incurring prohibitive costs or triggering provider countermeasures.
Theoretical Foundation: The paper includes a theoretical analysis (Appendix C) characterizing the sample efficiency of RUT under Zipfian assumptions of token distributions, showing that the required sample size scales inversely with the squared distance between distributions.

In conclusion, RUT represents a significant advancement in LLM API auditing, offering a robust, efficient, and stealthy mechanism to ensure the integrity of black-box model deployments.