Learning Nested Named Entity Recognition from Flat Annotations

1. Problem Statement

Nested Named Entity Recognition (Nested NER) involves identifying entities that are contained within other entities (e.g., "United Kingdom" inside "Ministry of Foreign Affairs of the United Kingdom"). While crucial for downstream tasks like relation extraction and knowledge graph construction, Nested NER suffers from a scarcity of training data because creating nested annotations is significantly more expensive and labor-intensive than creating flat (non-overlapping) annotations.

The authors address the following research questions:

• Can models learn to recognize nested structures using only flat annotations?
• Can Large Language Models (LLMs) bridge the gap between flat and nested supervision?
• Can LLMs compensate for missing nested annotations?

2. Dataset and Setup

• Dataset: The experiments utilize NEREL, a Russian benchmark containing 29 entity types (significantly more complex than typical 4–8 type datasets).
• Statistics: The dataset contains ~56k annotations, with 21% of all entity mentions being nested.
• Baseline: A Binder model (a bi-encoder span-based model using contrastive learning) built on RuRoBERTa-large.
• Evaluation: Micro F1 scores are reported separately for Overall, Inner (nested) entities, and Outer (top-level) entities.

3. Methodology

The paper investigates four complementary approaches to recover nested structure from flat data, categorized into data augmentation, training modifications, and LLM-based pipelines.

A. Data Augmentation (Creating Pseudo-Nested Signals)

1. String Inclusions:
   • Concept: Identifies subsequences within flat mentions that match the surface forms of other known entities in the corpus.
   • Mechanism: If a flat mention "Ministry of Foreign Affairs of Russia" exists, and "Russia" is a known entity, "Russia" is added as a positive nested example.
   • Enhancement: For Russian's rich morphology, lemmatized inclusions are used (matching canonical dictionary forms) to capture inflectional variants.
2. Entity Corruption:
   • Concept: Trains the model to recognize that parts of a long entity remain valid even if the entity is corrupted.
   • Mechanism: Words within long flat mentions are replaced with noise tokens (digits, random letters, "diglets," or punctuation). The model is trained to predict the remaining valid entity spans.
   • Finding: End-position corruption (replacing the last word) proved most effective, likely because the beginning of Russian noun phrases often carries the head word and type information.

B. Training Modifications

3. Flat Neutralization:
   • Concept: Addresses the "false negative" problem where valid nested spans are treated as negative examples during flat training.
   • Mechanism: Instead of treating all non-annotated spans as negative, the model introduces a neutral category. Spans that match known entity surface forms (via inclusion matching) are marked as "neutral" and excluded from the loss function, preventing the model from learning that they are definitely not entities.

C. LLM-Based Approaches

4. Pure LLM:
   • Uses few-shot prompting (DeepSeek-R1-32B and RuAdapt-Qwen2.5-32B) to extract all entities (nested and outer) directly.
   • Explores example selection strategies: Random, Most-Frequent-Entity (MFE), and MFE-entwise (balanced per type).
5. Hybrid Fine-tuned + LLM:
   • Pipeline: A fine-tuned Binder model first detects outer entities. Then, an LLM is queried specifically for inner entities within each detected outer span.
   • Goal: Leverage the boundary detection strength of fine-tuned models and the reasoning capabilities of LLMs for internal structure.

4. Key Results

• Gap Closure: The best combined weak supervision method (Lemmatized Inclusions + Corruption + Neutralization) achieves 26.37% Inner F1, closing 40% of the performance gap between flat training and full nested supervision.
• LLM Limitations: Pure LLM approaches performed poorly (42.39% Overall F1) compared to fine-tuned models. The Hybrid approach also underperformed the purely fine-tuned weak supervision methods on inner entities, suggesting current LLMs struggle with fine-grained nested structures across large type inventories (29 types).
• Corruption Strategy: End-position corruption with alphanumeric noise (letters/digits) yielded the best results among corruption strategies.

5. Key Contributions

1. Systematic Comparison: First comprehensive evaluation of methods (inclusions, corruption, neutralization) for learning nested NER from flat data on a challenging 29-type Russian benchmark.
2. Novel Corruption Analysis: Discovery that end-position corruption is the most effective strategy for generating pseudo-nested data, a finding not previously documented.
3. Hybrid Pipeline Evaluation: Introduction and evaluation of a two-stage Hybrid pipeline, demonstrating that while it improves over pure LLMs, it does not yet surpass fine-tuned weak supervision for nested detection.
4. Empirical Evidence: Proof that simple, architecture-agnostic methods can recover substantial nested structure without expensive gold annotations, closing 40% of the gap to full supervision.

6. Significance and Limitations

• Significance: The work provides a practical pathway for leveraging abundant flat NER corpora for nested tasks, significantly reducing the annotation barrier for complex entity recognition. It validates that "weak supervision" via data augmentation and training adjustments is a viable alternative to full nested annotation.
• Limitations:
  • Language Specificity: Experiments were conducted only on Russian (NEREL), which has rich morphology. Results may not generalize to languages with different structural properties.
  • Domain: Limited to news text; performance in biomedical or social media domains is unverified.
  • Architecture: Only the Binder architecture was tested; results may vary with biaffine parsers or sequence-to-sequence models.
  • LLM Capabilities: The study suggests current LLMs are not yet ready to fully replace supervised learning for fine-grained nested NER, particularly with large type inventories.

Conclusion: The paper concludes that while models can learn nested structures from flat annotations (answering RQ1), current LLMs cannot fully compensate for missing nested annotations (answering RQ3 negatively), and hybrid approaches offer only partial benefits (answering RQ2 partially). The most effective strategy remains a combination of data augmentation (inclusions/corruption) and training modifications (neutralization) applied to fine-tuned models.