Language Models Can Resolve Reference Compositionally, But It's Not Their Native Strength: The Case of the Personal Relation Task

Technical Summary: Language Models Can Resolve Reference Compositionally, But It's Not Their Native Strength

Problem Statement
The paper addresses a critical question in the evaluation of Large Language Models (LLMs): do these models genuinely acquire compositional abilities for interpreting natural language, or do their outputs result primarily from sophisticated pattern matching? While existing benchmarks often focus on either translating natural language into formal representations (Intensional tasks) or mapping descriptions to concrete referents (Extensional tasks), few studies explicitly contrast these two aspects of meaning within a unified framework. The authors argue that a lack of referential grounding in LLM training may be a crucial missing component in mimicking human-like language understanding, necessitating a method to distinguish between a model's ability to compute a "sense" (the structural formula) versus its ability to resolve a "reference" (the specific entity).

Methodology
The authors adapted the Personal Relation Task (PRT) (Paperno, 2022) to evaluate both LLMs and human participants. The task involves a controlled "universe" of six people connected by four types of relationships: friend, enemy, parent, and child. Participants are presented with complex noun phrases (e.g., "Amber's parent's friend") and asked to perform one of two tasks:

1. Extensional Task: Identify the specific person (referent) the phrase denotes within the provided universe.
2. Intensional Task: Generate a nested functional formula representing the sense of the phrase (e.g., friend(parent(amber))).

The experimental design manipulated four key variables:

• Approach: Extensional vs. Intensional.
• Representation: English (using standard names/relations) vs. Abstract (mapping names and relations to arbitrary variables, e.g., Amber $\to$ h, friend $\to$ x). The Abstract condition serves as a control for pretraining data contamination and increases the difficulty of the Intensional task by preventing superficial string manipulation.
• Branching: Left-branching (genitive 's, e.g., "Amber's parent's friend") vs. Right-branching (prepositional phrases, e.g., "the friend of the parent of Amber").
• Complexity: The number of relations involved plus one (tested at levels 3, 4, 5, and 6 for LLMs; limited to level 3 for humans due to budget constraints).

The study evaluated five LLMs, including standard instruction-tuned models and Large Reasoning Models (LRMs), using a standardized four-part prompt structure (introduction, dataset, worked example, test question). Human data was collected from 32 participants via Prolific. Performance was analyzed using Generalized Linear Mixed Models (GLMMs) to assess the effects of participant group, task approach, representation, and complexity.

Key Contributions

• Methodological Framework: The paper introduces a unified methodology for assessing intensional versus extensional compositional interpretation, allowing for direct comparison between these two distinct semantic capabilities.
• Comparative Data: It provides human performance data on the same tasks under maximally comparable conditions to LLMs, filling a gap where compositional abilities are often assessed without explicit human baselines.
• Empirical Evaluation: The study offers a statistical evaluation of several LLMs, highlighting divergent strengths between models and human participants, and specifically investigates the impact of reasoning-enhanced architectures.

Results
The study reveals a fundamental divergence in performance profiles between humans and LLMs:

• Opposite Strengths: Humans performed significantly better on Extensional tasks (82.8% accuracy) than Intensional tasks (71.1%). Conversely, LLMs performed better on Intensional tasks (95.0%) than Extensional tasks (79.8%).
• Representation Effects: Both groups found English representations easier than Abstract ones. However, the difficulty gap was most pronounced for LLMs in the Abstract Extensional condition (66.1% accuracy), which requires navigating the universe and mapping arbitrary symbols, whereas the English Intensional task (95% accuracy) was the easiest for LLMs, resembling a translation or syntactic rearrangement task.
• Complexity and Reasoning: LLM performance degraded as complexity increased. However, reasoning-enhanced models (LRMs) maintained a more stable performance gap between Extensional and Intensional tasks across complexity levels compared to standard models, suggesting that reasoning capabilities can mitigate, though not eliminate, the weakness in referential resolution.
• Branching: Unlike earlier recurrent neural network studies, modern Transformer-based LLMs did not show a significant systematic bias between left- and right-branching structures.

Significance and Claims
The authors claim that their results support the notion that the lack of referential grounding in LLM training is a critical deficit in achieving human-like language understanding. The findings suggest that LLMs excel at the "translation" of natural language into formal structures (Intensional tasks) because their training objective focuses on surface-form prediction and syntactic patterns. In contrast, humans, whose language use is grounded in real-world interaction, excel at identifying concrete referents (Extensional tasks) but struggle when forced to explicitly articulate the underlying structural formulas.

The paper posits that while LLMs can achieve high accuracy on compositional tasks, they do so through a different underlying process than humans—leveraging translation capabilities rather than genuine referential grounding. The authors conclude that arbitrary reference remains a noticeable challenge even for frontier models, reinforcing arguments that referential grounding is an important missing component to integrate into AI systems. The study is modest in its claims, noting that the experimental setup is somewhat artificial and that future work is needed to compare basic instruction-tuned models with reasoning-enhanced models more broadly.