💻 computer science

Calibrate What You SHIP: Post-Selection Risk Control for Verifier-Guided Text-to-Image Generation

This paper introduces SHIP, a method that calibrates release thresholds at the policy level rather than the candidate level to ensure valid risk control for verifier-guided text-to-image systems that employ test-time search, thereby significantly reducing the risk of released outputs while maintaining finite-sample validity.

Original authors: Xuanhua Yin, Shunqi Mao, Wei Guo, Chuanzhi Xu, Weidong Cai

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xuanhua Yin, Shunqi Mao, Wei Guo, Chuanzhi Xu, Weidong Cai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Modern computers have learned to paint pictures from words. You type a sentence like "a stone mushroom behind an elephant," and the machine generates an image that attempts to match that description. For a long time, these systems worked by creating a single image and showing it to you. But as the technology has improved, the process has become more like a search. Instead of making one picture and stopping, the computer might generate dozens of variations, check them against the instructions, discard the bad ones, and perhaps even rewrite the instructions to try again. It is a complex dance of creation and selection, guided by a digital judge that decides which images are good enough to show.

The problem is that we have been measuring the safety of these systems in the wrong way. Imagine a factory that produces thousands of widgets, but only ships the ones that pass a quality check. If you test the quality of every single widget that comes off the line, you get a certain idea of how good the factory is. But if the factory has a machine that picks only the best-looking widgets to ship, the pile of widgets you actually receive is different from the pile that was made. The standard way of testing these image generators has been to look at the individual pictures the computer makes, check if they are good, and set a rule for what gets sent out. The researchers in this study found that this approach is flawed. Because the computer is actively searching and choosing, the final image you see is not just a random sample; it is the result of a specific decision process. A rule that works for a single picture does not necessarily work for the final picture the system decides to release.

To solve this, a team of researchers from the University of Sydney introduced a new method called SHIP. Instead of testing the individual pictures the computer makes, they tested the entire decision-making process. They took a set of prompts—requests for images that the system had never seen before—and ran the complete system on them. This meant letting the computer generate its candidates, let its internal judge pick the best one, and then check if that final choice was actually good. They used a second, independent judge to evaluate the final image, ensuring the system wasn't just tricking its own internal checker. By running this full simulation many times, they could find the perfect setting for the system's safety rules. They looked for the most lenient rule that still kept the risk of showing a bad image below a specific limit.

The results showed that the old way of setting these rules was often too optimistic. When the researchers applied their new method to a popular image generator called FLUX, they found a significant difference. Using the old method, which looked at all the candidate images together, the system would release images with a risk of failure around 31 percent. When they used their new method to calibrate the entire process, they could lower that risk to just 16 percent while still showing images to the user. This did not mean the computer became better at drawing; it meant the system became much better at knowing when to stop and when to show a result. The researchers found that different search strategies—some that try many options quickly, others that refine a few options slowly—created different trade-offs. Some strategies were faster, while others covered more types of requests, but all of them needed this new kind of calibration to be trusted.

The study also revealed that the complexity of the request matters. When a prompt asked for many specific details, like counting objects or placing them in precise relationships, the system struggled more. The researchers noted that while the average risk could be controlled, some difficult requests still carried a high chance of failure, meaning the system might need to refuse to answer them entirely to stay safe. This is a crucial distinction: the goal is not to force the computer to answer every question, but to ensure that when it does answer, the answer is reliable. The team tested their method across different types of image generators and different search strategies, and in every case, checking the final output rather than the intermediate steps provided a clearer, safer picture of what the system could actually do.

This work changes how we think about reliability in artificial intelligence. It suggests that we cannot simply test the raw output of a machine and assume that the final product will be safe. We must test the entire process, including the choices the machine makes along the way. Just as a pilot does not just check the engine before a flight but also checks the navigation and the weather, we must check the whole journey of the image generator. By calibrating the final decision rather than the individual steps, we can build systems that are not only powerful but also trustworthy, knowing exactly how often they will succeed and when they should admit they cannot do the job.

Technical Summary: Calibrate What You SHIP

Problem Statement: The Candidate-to-Policy Calibration Mismatch

Modern text-to-image (T2I) systems increasingly rely on test-time search to select, refine, or stop among multiple generation candidates. These systems utilize verifier-guided inference to explore different noise seeds, rank drafts, rewrite prompts, or edit images before releasing a final output.

The core problem identified is a candidate-to-policy calibration mismatch. Traditional reliability methods calibrate verifier thresholds on individual, independently sampled candidate images. However, in deployment, the system does not release a generic "passing" candidate; it releases the specific image selected by a complex policy involving search, selection, and stopping mechanisms.

This search process alters the distribution of released outputs in two ways:

Prompt Reweighting: Search changes which prompts receive an output (e.g., sequential stopping accepts a prompt if any attempt passes, biasing the output distribution toward prompts with higher pass probabilities).
Within-Prompt Selection: Search changes which image is released for a given prompt (e.g., Best-of-N or refinement selects candidates based on imperfect online verifiers, altering the conditional loss distribution).

Consequently, controlling the risk of individual candidates (candidate-level calibration) does not guarantee control over the risk of the final released output (policy-level risk).

Methodology: SHIP (Selection-aware Held-out calibration of Inference Policies)

To address this mismatch, the authors introduce SHIP, a framework that calibrates the complete inference policy rather than individual candidates.

Core Mechanism

SHIP aligns the calibration unit with the release policy. Instead of evaluating pooled candidates, it treats each exchangeable observation as a held-out prompt paired with a complete policy rollout.

Replay and Evaluation: SHIP runs or exactly replays the complete deployed policy on a set of held-out prompts.
Independent Judging: For the image actually released by the policy, SHIP evaluates it using an independent target judge (a separate VLM), distinct from the online selector used during generation.
Threshold Selection: SHIP selects the most permissive threshold $\tau^*$ such that the upper confidence bound (UCB) of the released-output risk satisfies a prescribed risk budget $\alpha$ .

Atomic Verification and Risk Metrics

To localize failures, prompts are decomposed into atomic requirements (existence, count, attributes, spatial relations, actions).

Selector Risk ( $s$ ): Calculated using the online selector VLM (e.g., Qwen2.5-VL) via a Soft-TIFA geometric mean of atom scores.
Target Loss ( $\ell$ ): Calculated using an independent target VLM (e.g., Qwen3-VL) offline. The policy never observes this loss during generation.
Risk Metric: The geometric mean of atom scores is used because a single violated requirement can invalidate prompt following.

Validity and Certification

Finite-Sample Validity: For policies with a prespecified threshold grid, SHIP uses simultaneous confidence bounds (Bonferroni correction) to ensure that the selected threshold satisfies the risk budget with probability $1-\delta$ .
Replayability: For threshold-independent streams, a single cached trajectory can be replayed across multiple thresholds. For history-dependent policies (e.g., adaptive refinement), complete rollouts are required for each threshold, which is computationally more expensive but necessary for accurate evaluation.

Key Contributions

Formal Characterization of Mismatch: The paper decomposes the candidate-to-policy calibration mismatch into prompt reweighting and within-prompt selection, proving mathematically that candidate-level risk control ( $R_{img}$ ) does not imply policy-level risk control ( $R_{\pi}$ ).
The SHIP Framework: Introduction of a method that calibrates the entire release policy on held-out prompts, selecting the optimal threshold based on the actual distribution of released outputs.
Empirical Validation: Comprehensive evaluation across fixed, sequential, and adaptive T2I inference procedures (including tournament selection, refinement, prompt rewriting, and image editing) using multiple models (FLUX, Stable Diffusion 3, SDXL) and benchmarks (GenEval2, etc.).

Experimental Results

Experiments were conducted primarily on GenEval2 using the FLUX model with a target risk budget of $\alpha=0.30$ .

Risk Reduction: On GenEval2 with FLUX at $N=16$ , a pooled-candidate threshold yielded a released risk of 0.310. In contrast, SHIP reduced the released risk to 0.162 while maintaining valid coverage.
Coverage and Tradeoffs: SHIP recovers feasible operating points that balance risk, coverage, and compute. For instance, while Best-of-N rejection covers more prompts, it consumes more compute. SHIP allows for deliberate abstention to meet strict risk budgets.
Generalization: The method transfers across different backbones (SD3.5, SDXL) and tasks (image editing with FLUX.1-Kontext). In adaptive scenarios like RAISE refinement, SHIP reduced the test risk from 0.244 (pooled-candidate) to 0.208.
Human Audit: Human evaluation of released outputs confirmed the machine-learned improvements, showing a reduction in atomic loss from 0.112 to 0.076 and strict failure rates from 0.380 to 0.280.
Robustness: Across 200 cached-stream splits, the fixed-grid certificate showed no target crossing, demonstrating finite-sample validity.

Significance and Claims

The paper claims that reliable inference-time scaling requires calibrating the output distribution induced by the complete deployed policy, not just upstream candidates.

Distinction of Roles: The authors posit that search determines the attainable performance frontier (quality, coverage, cost), while calibration determines which points on that frontier can be safely released under a risk constraint.
Modest Scope: The paper acknowledges limitations:
- Validity relies on a prespecified grid, exchangeability, and bounded loss.
- Threshold-dependent adaptive search remains descriptive in some contexts because exact calibration may require a full rollout per prompt and threshold.
- The method controls average risk, not subgroup risk.
- The use of a held-out VLM for loss definition makes human audits corroborative rather than a formal human-risk certificate.
Conclusion: The primary contribution is the T2I-specific analysis of the estimand shift and the demonstration that policy-level calibration is necessary to ensure that the images actually delivered to users satisfy safety and quality constraints.