Technical Summary: Who Should Be Generated? Justifying Demographic Targets in Open-Ended Generation

1. Problem Statement: The Missing-Target Problem

The paper addresses a fundamental gap in the fairness evaluation of open-ended generative models (text and image). While existing fairness audits measure the demographic composition of model outputs against a target distribution, they typically assume the target is supplied or self-evident. This creates the missing-target problem: when a prompt specifies a role and a location but leaves demographic attributes (e.g., gender, race) unspecified, the model produces an empirical distribution ($\hat{q}_k$), but there is no justified standard ($P^*$) against which to evaluate it.

The authors argue that classical group fairness criteria (e.g., demographic parity, equalized odds) presuppose input-side sensitive attributes and decisions regarding existing individuals. These criteria do not select an aggregate output-side target for fictional generation where no ground-truth identity exists. Consequently, current evaluations often implicitly adopt the current occupational composition or equal-category distributions as targets without justifying why those specific distributions should govern the evaluation.

2. Methodology: A Four-Commitment Framework

The authors formalize target construction as a distinct evaluative task requiring four logically distinct commitments. This framework separates the construction of a target from the verdict of fairness.

2.1 The Four Commitments

1. Evaluative Object, Use, and Output Unit:

   • Object: The distribution of representational possibilities across repeated generations, not the accuracy of predicting a specific real person.
   • Use: For the study, a "contemporary public-world use" is declared, asking how generations distribute the possibility of being a role-bearer in a named social world.
   • Unit: Each generated focal character is one output observation.

2. Prior Admissibility (Relationship Identification and Defense):

   • A demographic prior (e.g., population composition, workforce composition) is only admitted if the evaluator identifies the relationship under which it is proposed and defends why that relationship grants the identified persons representational standing.
   • Geographic Membership: Admitted for public-world use. The prompt's location identifies the represented public; the role does not restrict who counts as a member of that public.
   • Occupational Incumbency: Not admitted for public-world use under the incumbency interpretation. Current workforce composition is a downstream outcome of labor-market processes (education, discrimination, barriers) and does not self-justify as a representational standard without an independent objective (e.g., workforce simulation).

3. Allocation:

   • Once a domain is admitted, a rule must determine how representational mass is divided.
   • Equal-Person Allocation: In the absence of a defended allocation-relevant distinction, members of the admitted public receive equal weight. This leads to category shares proportional to the population shares of the admitted public.
   • Role Invariance: Under this construction, the target is role-invariant ($P^*(A=a|G,R) = P^*(A=a|G)$) because the role does not alter the admitted public or the allocation rule.

4. Operationalization:

   • The abstract target is instantiated using a declared taxonomy, mapping, and reference source (e.g., resident population data from census or UN statistics).
   • The paper distinguishes between admissibility (normative justification) and measurement adequacy (whether the source proxy accurately measures the admitted public).

2.2 The Bridge Premises

The paper distinguishes between the operational score (the computed discrepancy between $\hat{q}_k$ and the constructed target $P^*_{geo}$) and the fairness estimand (the theoretical divergence from an exhaustively justified target $P^\dagger$).

• Premise A: Compositional unfairness is defined as the divergence of the model's limiting composition from the exhaustively justified target.
• Premise B: The operational score adequately estimates this fairness estimand. The paper acknowledges this requires three substitutions (finite generation, source proxy, and non-exhaustive construction space) and does not claim these are fully resolved.

3. Experimental Setup: AP-Bench

To instantiate this framework, the authors introduce AP-Bench, a benchmark comprising 51,840 generations from six frontier models (GPT-5.6, Claude Sonnet 5, Gemini 3.5 Flash, Kimi K3, Llama 4 Maverick, Qwen3.7-Max).

• Scope: 12 geographic contexts, 6 roles (CEO, doctor, nurse, engineer, teacher, cleaner), and 30 samples per cell.
• Attributes: Race/ethnicity, religion, sexual orientation, and a gender-to-sex diagnostic.
• Prompting: Both implicit prompts (demographics unspecified) and joint-elicitation prompts (demographics explicitly requested but values unspecified).
• Target Construction: Admits the geographic prior under a geographic-membership interpretation and applies equal-person allocation using resident population data as the operational source.

4. Key Results

4.1 Substantial Divergence from Geography-Derived Targets

Under joint elicitation, models exhibit significant divergence from the constructed geography-derived targets.

• The composite three-module Jensen-Shannon Divergence ($JSD^2$) ranges from 0.508 to 0.606 across models.
• This divergence is far larger than what can be explained by finite-sample variation (upper 99% reference endpoint $\approx 0.033$).

4.2 Sensitivity to Comparator Choice

The paper demonstrates that fairness evaluations are highly sensitive to the choice of target.

• Replacing the geography-derived target with an equal-category comparator (assigning equal probability to each category regardless of population size) while holding generations and measurement fixed produces substantial changes in scores.
• The mean absolute cell-level change in $JSD^2$ ranges from 0.279 to 0.355.
• This shift alters model rankings and the classification of cells as "fair" or "unfair" under various tolerance thresholds (12.8% to 40.0% of cells change classification).

4.3 Occupational vs. Geographic Targets

In a diagnostic analysis of U.S. CEO generations:

• Models generated a high share of female/woman characters (77%–100%).
• Compared to the occupational incumbency target (33.0% female), this appears as a "success" (closer to parity).
• Compared to the geographic membership target (49.8% female), the divergence is significant.
• The paper argues that adopting the occupational target implicitly commits the evaluation to fidelity to historical labor-market outcomes, which requires a separate defense (e.g., workforce simulation) not present in the declared public-world use.

5. Significance and Claims

The paper makes modest but foundational claims regarding the state of fairness evaluation:

1. Target Construction is Part of Evaluation: Target construction is not a preliminary step but a core component of fairness evaluation. Supplying a target without justification is insufficient.
2. No Universal Target: The paper does not provide a universal target. Instead, it provides a framework of justification (the four commitments) that makes explicit the normative assumptions required before a distribution can serve as a standard.
3. Rejection of Default Targets: It argues that occupational incumbency cannot serve as a default target for public-world representation without an independently defended objective (like workforce fidelity). Geographic membership is admitted under a "represented-public" premise for the declared use.
4. Burden of Proof: The primary contribution is shifting the burden of proof. Evaluators must explicitly answer: What is distributed? Whose representation counts (and why)? How is mass allocated? Which source instantiates the target?
5. Distinction from Harm: The paper clarifies that measuring divergence from a constructed target is a diagnostic step, not a direct measure of representational harm. Establishing harm requires additional premises about exposure, mechanism, and magnitude.

The authors conclude that until evaluators can explicitly justify why a specific distribution deserves to be the standard, fairness evaluations remain incomplete. The framework allows for the rejection of specific targets (e.g., occupational) while leaving open the possibility of other defended constructions (e.g., corrective targets) provided their premises are explicitly stated.