Technical Summary: Every Act Has Its Price: Compressed Moral Composition in Frontier LLMs

Problem Statement

Existing moral benchmarks for Large Language Models (LLMs) predominantly rely on isolated act rankings. In these standard audits, models choose between two options, each containing a single focal moral act. While this approach effectively establishes preference hierarchies for individual acts or moral foundations (e.g., Care, Fairness), it fails to capture how models process composite moral evidence.

Real-world moral decision-making often involves comparing options where multiple moral signals coexist within a single choice (e.g., an option combining an extreme positive act with a mild violation versus an option with two moderate positive acts). Current methodologies cannot determine whether models simply sum these signals, discount secondary components, or allow the strongest act to dominate the entire option. Consequently, a model's ranking of isolated acts does not necessarily predict its judgments when those acts are composed. This paper addresses the gap in understanding how frontier LLMs compose multiple calibrated moral acts into composite judgments.

Methodology: MORAL TROLLEY ARENA

To measure moral composition, the authors introduce MORAL TROLLEY ARENA, a two-stage blind ELO benchmark designed to link act-level calibration with composite judgment. The framework utilizes the Moral Foundations Theory (MFT) covering five foundations: Care, Fairness, Loyalty, Authority, and Sanctity.

1. Scenario Construction

The benchmark utilizes a corpus of 229 person-centric moral scenarios derived from established psychological vignettes (Clifford et al., 2015; Young & Saxe, 2011; Hofmann et al., 2014) and augmented with anchor cases. Each scenario is assigned a primary MFT foundation and a semantic intensity label ranging from $-2$ (extreme wrongdoing) to $+2$ (exceptional virtue).

2. Two-Stage Arena Protocol

The benchmark operates through two linked arenas using a blind pairwise ELO update protocol (similar to Chatbot Arena):

• Stage A: Single-Scene Calibration

  • Goal: Establish a baseline "component strength" for each atomic act.
  • Process: The 229 scenarios are evaluated in isolation via blind pairwise battles. Foundation labels are hidden from the model.
  • Output: A calibrated ELO score for each scenario, representing its moral weight independent of composition.

• Stage B: Composite Arena

  • Goal: Measure how models trade off multiple acts when presented together.
  • Process: Calibrated acts are combined into two-act moral items. The authors constructed a controlled grid of 160 composite profiles by crossing all $\binom{5}{2}=10$ foundation pairs with 16 non-neutral intensity configurations ($\{-2, -1, +1, +2\}^2$).
  • Protocol: Models judge 1,200 composite-vs-composite battles. Foundation labels remain hidden.
  • Output: A composite ELO score for each profile, reflecting the model's integrated judgment.

3. Analytical Framework

The authors analyze the relationship between the Single-Scene ELOs (component strength) and Composite ELOs using four specific signals:

1. Compressed Composition Rule: Testing if composite ELO is a linear function of component ELOs ($E_c = \beta(E_{s1} + E_{s2}) + \alpha$).
2. Foundation Residuals: Measuring deviations from the linear prediction after controlling for component strength to identify foundation-specific weighting.
3. Intensity Anchoring: Investigating non-additive effects where specific intensity configurations (e.g., $+2, -1$) yield different results than others with matched component sums (e.g., $+1, +1$).
4. Cross-Model Convergence: Assessing the correlation of composite preference surfaces across different model providers.

Key Contributions

1. MORAL TROLLEY ARENA Benchmark: A novel two-stage blind ELO benchmark that shifts moral auditing from isolated act rankings to judgments over composed moral evidence.
2. Characterization of Composition Rules: The authors identify that composite judgments follow a compressed linear function of component strengths rather than a simple additive model.
3. Discovery of Intensity Anchoring: The study reveals that models exhibit non-additive intensity anchoring, where profiles with an extreme positive component and a mild negative component ($+2, -1$) outperform profiles with two moderate positive components ($+1, +1$), despite matched component strength sums.
4. Foundation-Specific Residuals: After controlling for component strength, the authors isolate bounded residuals, finding a positive residual for Loyalty and a negative residual for Care, while other foundations align closely with linear predictions.
5. Cross-Provider Convergence: The composite preference surfaces show high convergence (mean pairwise $r = 0.939$) across ten frontier models from nine different providers, suggesting these composition behaviors are shared properties of current frontier models.

Results

The study evaluated ten frontier models (including Claude-Sonnet-4.6, GPT-5.2, Gemini-2.5-Pro, etc.), generating a corpus of 29,134 blind pairwise judgments.

• Compressed Integration: Across all models, the slope ($\beta$) of the linear relationship between component ELO sums and composite ELOs was consistently less than 1 ($\bar{\beta} = 0.862$). This indicates that models compress combined moral evidence rather than summing it linearly.
• Intensity Anchoring: A significant deviation from monotonicity was observed. Profiles with an intensity sum of $s=+1$ (dominated by a $+2/-1$ pair) received higher composite ELOs ($1098.4$) than profiles with $s=+2$ (dominated by $+1/+1$ pairs, $1063.4$), despite having nearly identical component ELO sums. This suggests the presence of a high-intensity anchor drives the judgment more than the aggregate sum.
• Foundation Residuals: When controlling for component strength, Loyalty exhibited a positive residual ($+9.58$), indicating it receives extra weight in composite judgments. Conversely, Care showed a negative residual ($-9.22$).
• Convergence: The 160-dimensional composite ELO vectors across models were highly correlated (mean $r=0.939$), indicating that the elicited composition behavior is robust across different model architectures and training pipelines.

Significance and Claims

The paper argues that moral audits must evolve beyond checking isolated act rankings to measuring composition rules. The findings suggest that:

• Simple rankings are insufficient: A model's preference for isolated acts does not fully predict its behavior when multiple moral signals are present in the same option.
• Composition is non-linear: Models do not simply add moral values; they compress evidence and are influenced by intensity anchoring.
• Shared Behavioral Primitives: The consistency of these composition rules across diverse frontier models implies that these are fundamental properties of current LLM architectures, which has implications for how moral evidence accumulates, dominates, or cancels out in complex agent societies and bias propagation pipelines.

The authors maintain a modest scope, noting that the benchmark measures revealed choices under forced trolley framing and does not constitute a full theory of model morality. They acknowledge limitations regarding the 5-level intensity scale, the use of LLM-labeled scenarios for intensity, and the exclusion of smaller open-weight models and multilingual settings.