💻 computer science

MM-SCALE: Grounded Multimodal Moral Reasoning via Scalar Judgment and Listwise Alignment

This paper introduces MM-SCALE, a large-scale dataset and framework that enhances Vision-Language Models' moral reasoning by replacing discrete binary supervision with continuous 5-point scalar ratings and grounded listwise alignment to better capture the nuanced, pluralistic nature of human moral judgments.

Original authors: Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin, Matheus Kunzler Maldaner, Jordan Wheeler, Jason I. Hong, Hong Shen, Adam Perer, Ken Holstein, Motahhare Eslami, Gunhee Kim

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Eunkyu Park, Wesley Hanwen Deng, Cheyon Jin, Matheus Kunzler Maldaner, Jordan Wheeler, Jason I. Hong, Hong Shen, Adam Perer, Ken Holstein, Motahhare Eslami, Gunhee Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to be a "good person" when it looks at pictures and reads stories.

The Problem: The Robot is Too Black-and-White
Currently, most robots (called Vision-Language Models) are trained to see the world in strict "Good vs. Bad" boxes. If you show them a picture of someone helping a stranger, they say "Good." If you show them someone pushing a stranger, they say "Bad."

But real life isn't a black-and-white movie; it's a full-color spectrum. Sometimes, "helping a stranger" is great (like giving someone a ride in the rain). But other times, it's risky (like giving a ride to a stranger late at night in a sketchy neighborhood). The robot struggles to see these subtle shades of gray. It treats all "helping" as equally good, missing the context that makes a situation dangerous or acceptable.

The Solution: MM–SCALE
The researchers created a new tool called MM–SCALE (Multimodal Moral Scale). Think of this as a giant, high-definition training manual for robots, but with two special features:

The 1-to-5 Star Rating (Scalar Judgment): Instead of just asking "Is this good or bad?", they asked humans to rate scenarios on a scale of 1 to 5.
- Analogy: Imagine a restaurant review. A binary system only lets you say "Eat it" or "Don't eat it." MM–SCALE lets you say, "It's okay, but the soup was cold," or "It's amazing, but a bit spicy." This teaches the robot that some actions are mostly good, some are mostly bad, and some are in the middle.
The "Why" Tag (Grounded Reasoning): The researchers also asked humans to point out why they gave a certain rating. Did they decide based on the text (the story), the image (what they see), or both?
- Analogy: Imagine you are judging a magic trick. If you say "That was fake," are you saying that because the story said it was a trick, or because you saw the hidden wire in the picture? MM–SCALE teaches the robot to know which clue (picture or words) is actually driving the decision.

How They Built It: The "MORALE" Interface
To create this dataset, the team built a special website called MORALE.

They generated thousands of images of social situations (like a child running into traffic).
They wrote different story scenarios for each image (e.g., "The child is playing tag" vs. "The child is running toward a car").
Humans looked at the image and the story, gave it a 1–5 rating, and tagged whether the picture or the words mattered more.
The "Disagreement" Loop: The system also showed the robot's guess. If the robot guessed "Safe" but the human said "Unsafe," the system flagged it. This forced the human to double-check why they disagreed, helping the researchers find the tricky cases where the robot was confused.

The Results: A Smarter, More Nuanced Robot
When they trained robots using this new MM–SCALE data, the results were impressive:

Better Ranking: When asked to sort a list of scenarios from "Most Safe" to "Least Safe," the robots trained with MM–SCALE did a much better job than those trained on old "Good/Bad" data. They could tell the difference between "slightly risky" and "very dangerous."
Stability: The robots didn't just guess randomly; their confidence levels matched reality better.
The "Image" Surprise: The study found that 68% of the time, humans changed their moral judgment after seeing the picture. For example, a text might say "Helping someone," which sounds good. But if the picture shows the person is actually being pushed into traffic, the human changes their mind. The new training method taught the robot to pay attention to that visual clue.

In a Nutshell
The paper argues that to make AI morally smart, we can't just teach it "Right vs. Wrong." We have to teach it the spectrum of right and wrong, and show it how the visual world changes the meaning of a situation. MM–SCALE is the first big dataset to do this, helping robots understand that context is everything.

Technical Summary: MM–SCALE

Problem Statement

Vision-Language Models (VLMs) currently struggle to make morally salient judgments in multimodal and socially ambiguous contexts. Existing safety alignment methods typically rely on binary (safe/unsafe) or pairwise (A vs. B) supervision. These approaches fail to capture the continuous, scalar nature of human moral reasoning and often overlook how visual context and situational nuances reshape moral interpretations. Prior benchmarks lack coverage on scalar judgments and explicit modality grounding (whether a judgment depends on text, image, or both), limiting the ability of models to handle pluralistic, context-sensitive moral reasoning.

Methodology

1. Dataset Construction: MM–SCALE

The authors introduce MM–SCALE (MULTIMODAL MORAL SCALE), a large-scale dataset comprising 32,212 annotated moral scenarios grounded in AI-generated images.

Source Material: Scenarios are derived from the Commonsense NormBank, visualized using Stable Diffusion v1.5 and DALL·E 3.
Annotation Schema: Each image-scenario pair is annotated with:
- Scalar Moral Ratings: A 5-point scale (1–5) for moral acceptability.
- Modality Grounding: Labels indicating whether the judgment is based on the text, the image, or both.
Annotation Pipeline (MORALE): The authors developed MORALE (MORal Alignment and Listwise Evaluation), a web-based interactive interface.
- Model-in-the-Loop: The system compares annotator ratings with VLM predictions.
- Discrepancy Handling: If a model's judgment deviates significantly from the human rating, the system triggers a modality grounding check to determine if the model attended to the wrong modality.
- Expansion: If the model and human agree, annotators are prompted to add new, image-relevant scenarios to expand coverage.
Quality Control: The dataset underwent rigorous filtering, including variance screening and "canary checks," resulting in high inter-annotator consistency (Krippendorff's $\alpha$ = 0.74 for scores, 0.71 for modality).

2. Training Framework: Listwise Preference Optimization

Instead of treating scalar judgments as independent labels or reducing them to binary classes, the authors employ a listwise preference optimization framework.

Objective: The model is trained to reproduce the human preference ordering over a full set of scenarios associated with a single image.
Loss Function: The authors utilize ListMLE (Listwise Maximum Likelihood Estimation) loss. This encourages the model to assign scalar scores such that the resulting ranking matches the ground-truth permutation of human ratings.
Architecture: The framework fine-tunes a scalar regression head on top of pre-trained VLMs (e.g., LLaVA-OneVision, Qwen2-VL, Phi-3-Vision, InstructBLIP).

Key Contributions

MM–SCALE Dataset: A novel dataset of 32,212 multimodal moral scenarios featuring scalar (1–5) ratings and explicit modality grounding annotations, distinguishing it from prior binary or pairwise benchmarks.
Listwise Alignment Analysis: An analysis demonstrating how scalar supervision combined with multimodal grounding improves VLM alignment with human moral preferences, moving beyond isolated binary comparisons.
MORALE Interface: An interactive, open-source web interface designed to collect disagreement data at scale, facilitating human-centered alignment and dataset expansion through model-in-the-loop feedback.

Experimental Results

The authors evaluated VLMs fine-tuned on MM–SCALE against models trained with binary preference (BPO) and binary classification (BCE) objectives.

Ranking Fidelity: Models trained with listwise scalar supervision consistently achieved the highest ranking metrics, including NDCG@5 (up to 0.89) and Mean Reciprocal Rank (MRR), outperforming binary and pairwise baselines. This indicates that learning full ordinal structure provides better global moral ordering.
Safety Calibration: While binary-trained models showed slightly lower "Unsafe Rates" (a binary metric favoring threshold-based objectives), they exhibited poorer AUC-Safety and less stable calibration. Listwise models produced smoother, more discriminative scalar predictions, maintaining competitive safety rates while offering superior threshold-free risk estimation.
Modality Grounding: Empirical analysis revealed that 68% of human moral judgments shifted after viewing the image compared to text-only labels. Furthermore, 78% of these divergent judgments were grounded in the image or a combination of image and text, confirming the necessity of multimodal grounding.
Human Consistency: Listwise-supervised models showed the strongest agreement with human consensus (Kendall's $\tau$ = 0.68) compared to pairwise (0.60) and binary (0.55) approaches.

Significance and Claims

The paper claims that MM–SCALE addresses a critical gap in multimodal safety alignment by shifting from discrete, binary supervision to scalar, listwise supervision with explicit modality grounding.

Nuance over Binary: The authors argue that moral alignment is not solely about the action but also the context in which it unfolds. Scalar ratings capture fine-grained acceptability and ambiguous, partially acceptable actions that binary labels miss.
Contextual Sensitivity: The dataset demonstrates that visual context frequently reconfigures moral interpretation, with significant shifts in judgment occurring when images contradict or reinforce text.
Stability: The study highlights that scalar listwise supervision produces more consistent moral rankings across modalities and maintains stability across synthetic and real images, suggesting that supervision granularity is key to improving moral alignment.

The authors position MM–SCALE as a foundation for future work on socially aligned multimodal models, particularly for tasks requiring the resolution of situational ambiguities through both language and vision. They acknowledge limitations regarding the cultural diversity of annotators (primarily US/UK) and the coarse nature of modality attribution (text/image/both), suggesting these as areas for future refinement.