Where Norms and References Collide: Evaluating LLMs on Normative Reasoning

Technical Summary: Evaluating LLMs on Normative Reasoning

Problem Statement
Embodied agents, such as robots, operate in situated environments where successful communication often relies on Norm-Based Reference Resolution (NBRR). This process requires interpreting referential expressions (e.g., "pass me the mug") by inferring implicit social norms grounded in physical and social context (e.g., selecting a clean mug over a dirty one in a serving context). While Large Language Models (LLMs) demonstrate strong capabilities in general commonsense reasoning and question answering, it remains unclear whether they can reliably identify and apply these specific, physically grounded norms, particularly when they are implicit, underspecified, or in conflict. Existing datasets for reference resolution (e.g., Winograd Schema, WinoGrande) or social norm modeling (e.g., Social Chemistry 101) fail to integrate the necessary physical constraints and situational nuances required for NBRR.

Methodology
To address this gap, the authors introduce SNIC (Situated Norms in Context), a human-validated diagnostic testbed designed to probe LLMs' ability to extract and utilize normative principles for reference resolution.

1. Dataset Construction:

   • Seed Generation: The process began with 120 handcrafted scenarios grouped into nine categories of norm reasoning (e.g., serving with clean items, prioritizing hazard removal).
   • Human Validation: An online study with 210 participants validated the initial set. Participants selected referents in ambiguous scenarios and provided justifications. The authors filtered the data to retain only 51 "seed" questions where the norm-guided referent was the plurality choice (most selected) among participants.
   • Procedural Augmentation: Using a rule-based augmentation strategy, the 51 validated seeds were expanded into a dataset of 9,000 examples. This process varied object types, properties (e.g., clean vs. dirty), settings (e.g., kitchen, library), and tasks (e.g., cleaning, serving) while maintaining the core normative structure of the seed scenarios.
   • Format: The dataset includes natural language vignettes and, for some evaluations, a formal Prolog (First-Order Logic) representation to eliminate ambiguity regarding object identities and properties.

2. Evaluation Protocol:
   The authors evaluated a diverse set of state-of-the-art LLMs (including Granite, Phi, Llama-3, Qwen, and GPT-4 variants) under three distinct conditions:

   • Description Only: The model receives the natural language scene description.
   • Description + Formalization: The model receives the scene description plus a Prolog formal specification of the scene.
   • Description + Formalization + Explicit Norms: The model receives the scene, the formalization, and an explicit list of the social norms to follow.

Key Results

• Baseline Performance: Without explicit guidance, LLMs struggle to consistently identify the correct referent based on implicit norms. The average accuracy across all models in the "Description Only" and "Description + FOL" settings was approximately 44%, with the best-performing model (GPT-4.1) achieving roughly 59.62% in the formalized setting.
• Impact of Formalization: The addition of Prolog formalization did not significantly improve performance across the board. In some cases (e.g., Llama-3 3B), performance degraded, suggesting models may lack the ability to leverage formal logic to resolve social ambiguities or that the formalization does not compensate for a lack of normative knowledge.
• Impact of Explicit Norms: When the specific social norms were explicitly provided in the context, model performance increased significantly, with the average accuracy rising to 70.51%. GPT-4.1 achieved 99.6% accuracy in this setting. This indicates that while LLMs can follow norms when instructed, they lack the inherent ability to infer them from context alone.
• Norm Conflicts: Analysis of correlations between norm categories revealed that models often prioritize certain tasks (e.g., tidiness) over others (e.g., safety) inconsistently, suggesting a lack of robust understanding regarding the deontic force (priority) of conflicting norms.

Key Contributions

• SNIC Dataset: The creation of a large-scale (9,000 instances), human-validated dataset specifically designed for evaluating norm-based reference resolution in physically grounded contexts.
• Diagnostic Framework: A controlled evaluation methodology that isolates the ability of LLMs to handle implicit norms, norm conflicts, and physical grounding, distinguishing this from general commonsense reasoning.
• Empirical Evidence: Demonstration that current LLMs exhibit a "blind spot" in normative reasoning, struggling to apply implicit social constraints without explicit prompting, even when provided with formal scene representations.

Significance and Claims
The paper claims that SNIC reveals a critical limitation in contemporary language models: their inability to consistently reason about physically situated social norms. The authors argue that while LLMs excel at tasks involving frequently stated facts, they struggle with norms that are implicit, culturally shared, or context-dependent. This gap poses a significant challenge for deploying language-based systems in socially situated, embodied settings (e.g., robotics), where agents must resolve ambiguities based on shared expectations rather than explicit instructions. The work underscores the need for further alignment and training strategies that specifically target the integration of implicit normative knowledge, rather than relying solely on the scale of pre-training data. The authors remain modest, noting that their text-based testbed does not evaluate multimodal reasoning or physical affordances, and that the rapid evolution of LLMs makes broad generalizations about all state-of-the-art models difficult.