Technical Summary: Revisiting the Systematicity in Negation in the Era of In-Context Learning

Problem Statement

Despite the capabilities of Large Language Models (LLMs), understanding negation remains a significant challenge. While models often perform well on specific benchmarks, their ability to generalize systematically—comprehending an unbounded set of structured expressions from a finite set of primitives—is questionable. Previous evaluations have largely focused on behavioral systematicity (performance on held-out inputs), often failing to distinguish between genuine compositional understanding and spurious correlations or memorization.

Furthermore, while representational systematicity (the internal encoding of compositional structure) is theoretically crucial for systematic generalization, it has been under-explored in the context of In-Context Learning (ICL). Most prior work on representational analysis has focused on pretrained models like BERT, leaving the systematicity of ICL-based LLMs insufficiently explored. This paper addresses the gap by analyzing both behavioral and representational systematicity regarding LLM understanding of negation.

Methodology

The study employs a dual-perspective approach:

1. Behavioral Systematicity Analysis

The authors evaluate LLMs under a few-shot learning setting to determine if models can recognize negation expressions and their scopes in sentences containing word combinations not seen in the demonstrations.

• Tasks:
  • Negation Cue Extraction: Identifying the negation expression (e.g., "not").
  • Negation Resolution: Predicting the scope of the negation (the part of the sentence affected by the cue).
  • Baseline: Antonym prediction (used to verify if function vectors can be constructed for simpler tasks).
• Output Formats: To test robustness, the negation resolution task was evaluated across four distinct formats:
  • EXTRACT: Outputting the token sequence within the scope.
  • BRACKET: Enclosing the scope in square brackets.
  • LAST-WORD: Outputting the final token of the scope.
  • NER: A sequence labeling task (0 for outside scope, 1 for inside).
• Models: GPT-J-6B and Qwen3-4B.
• Dataset: A subset of the SFU Review Corpus, filtered for sentences with negation cues and scopes, ensuring training, development, and test sets contain distinct word combinations.

2. Representational Systematicity Analysis

The authors investigate whether the internal mechanisms of LLMs encode the task of negation understanding as a robust, reusable vector representation.

• Function Vectors: Dense vectors extracted from the model's hidden activations (specifically from a subset of attention heads identified via causal mediation analysis) that summarize the input-output mapping demonstrated by in-context examples.
• Task Vectors: Vectors extracted directly from the residual stream activations of a designated task-representative token position.
• Intervention: These vectors are injected into the model during inference (zero-shot settings) to see if they can induce the target task behavior without explicit demonstrations.
• Evaluation: The study measures the zero-shot top-1 accuracy achieved when these vectors are applied to models, specifically focusing on test examples where the models performed correctly in a 10-shot setting (to isolate the vector's effect from general model capability).

Key Results

Behavioral Systematicity

• Negation Cue Extraction: Both models achieved accuracy comparable to the antonym prediction task (approx. 62–64%), suggesting they can partially learn to identify negation cues from demonstrations. However, performance was not perfect, indicating a lack of complete systematic ability.
• Negation Resolution: Performance varied significantly by output format. Models struggled notably with the NER format (0% accuracy for both models in the 10-shot setting) and the LAST-WORD format. The EXTRACT format yielded better results (up to 76% for Qwen3-4B), but overall, models failed to achieve perfect performance across formats, suggesting difficulty in generalizing the concept of "scope" across different output structures.

Representational Systematicity

• Negation Cue Extraction: Function vectors (applied in middle layers) and task vectors (applied in late layers) successfully induced task performance in zero-shot settings, achieving high accuracy (90% for GPT-J-6B, 80% for Qwen3-4B). This indicates that the task of identifying negation cues can be robustly encoded as a function vector.
• Negation Resolution: In contrast, function and task vectors had a significantly weaker effect on negation resolution tasks (0–50% accuracy). While there was a slight steering effect for EXTRACT and LAST-WORD formats in GPT-J-6B, the vectors generally failed to robustly induce the task.
• Vector Comparison: Task vectors appeared more effective than function vectors for negation resolution tasks, particularly in later layers.

Significance and Claims

The paper claims that while LLMs can learn to recognize negation cues through in-context learning, they struggle to achieve perfect behavioral systematicity, particularly when the output format changes. More critically, the representational analysis reveals a divergence in how these tasks are encoded internally:

1. Cue Extraction vs. Scope Recognition: The models can form robust function vectors for extracting negation cues, similar to simpler tasks like antonym prediction. However, constructing function vectors for recognizing negation scope is significantly more challenging.
2. Limitations of ICL: The difficulty in forming robust vectors for scope recognition suggests that LLMs may not possess the structured internal representations necessary for systematic inference regarding negation scope, even when they can produce correct outputs in specific few-shot contexts.
3. Methodological Contribution: The study highlights the necessity of moving beyond behavioral benchmarks to representational analyses (using function/task vectors) to truly assess whether LLMs have acquired the underlying principles of systematic generalization.

The authors conclude that while LLMs show some capacity for systematic behavior in negation tasks, the inability to robustly construct function vectors for scope recognition points to a fundamental limitation in their representational systematicity for this specific linguistic phenomenon.