💬 NLP

ToMMeR -- Efficient Entity Mention Detection from Large Language Models

ToMMeR is a lightweight model that demonstrates mention detection naturally emerges from language modeling and achieves competitive Named Entity Recognition performance by efficiently recovering structured entity representations from early transformer layers with minimal parameters.

Original authors: Victor Morand, Nadi Tomeh, Josiane Mothe, Benjamin Piwowarski

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Victor Morand, Nadi Tomeh, Josiane Mothe, Benjamin Piwowarski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, super-smart library assistant (a Large Language Model, or LLM) who has read almost everything on the internet. This assistant is incredibly good at writing stories, answering questions, and chatting. But, if you ask it to "find all the names of people and places in this text," it often gets confused, misses things, or makes up facts. It's like asking a brilliant novelist to act as a librarian; they have the knowledge, but they aren't using the right tool for the job.

This paper introduces ToMMeR, a tiny, super-efficient "add-on" that turns that brilliant novelist into a lightning-fast, hyper-accurate librarian.

Here is the breakdown using simple analogies:

1. The Problem: The "Goldilocks" Bottleneck

In the world of AI, finding specific things in text (like "Marie Curie" or "Tesla Inc.") is called Entity Mention Detection.

The Old Way: We used to build huge, heavy robots (massive AI models) trained specifically to find these names. They were slow, expensive, and only knew the specific rules they were taught. If you asked them to find a new type of thing, they failed.
The New Problem: We realized that the giant LLMs (the "novelists") already know how to find these names deep inside their brain, but they don't know how to "speak" that information out unless we force them to write a whole new essay about it.

2. The Solution: ToMMeR (The "Flashlight")

The authors created ToMMeR. Think of ToMMeR not as a new brain, but as a tiny, specialized flashlight (less than 300,000 parameters—tiny compared to the billions in a standard AI).

How it works: Instead of making the giant AI write a new story, ToMMeR just shines its flashlight on the early layers of the AI's brain while it's reading a sentence.
The Analogy: Imagine the AI is a giant, complex factory. The "mention detection" (finding names) happens on the very first assembly line, but the factory usually ignores it and keeps processing the product. ToMMeR is a small sensor attached to that first line that says, "Hey! That looks like a name! I'll flag it!"
The Result: It finds names 42 times faster than asking the AI to chat about them, and it catches almost everything (93% recall).

3. The "Aha!" Moment: Everyone Sees the Same Thing

The researchers did something fascinating. They tested ToMMeR on many different AI models, ranging from tiny ones (14 million parameters) to huge ones (15 billion parameters).

The Discovery: No matter which AI model they used, the "flashlight" found the exact same boundaries for names.
The Metaphor: It's like asking 100 different people (from a child to a professor) to draw a circle around the word "Apple" in a sentence. Even though they are different people, they all draw the circle in the exact same spot.
Why this matters: This proves that finding names isn't a special skill we have to teach AI. It's a natural byproduct of learning language. As soon as an AI learns to predict the next word in a sentence, it automatically learns where the "things" (entities) begin and end.

4. The "Judge" Problem: How do we know it's right?

Usually, we check if an AI is right by comparing it to a "Gold Standard" list (like a textbook answer key). But these answer keys are often incomplete. They might miss nested names or weird concepts.

The Solution: The authors used a "Judge" (another AI, GPT-4) to review the work. They asked the Judge: "Is this a real thing, even if it's not in the textbook?"
The Result: ToMMeR was incredibly accurate. It rarely made up fake names. It found things the textbooks missed, proving it understands the concept of an entity, not just a list of rules.

5. Why Should You Care?

Speed & Cost: ToMMeR is so small and fast that it can run in real-time. You could plug it into a live chat or a news feed to instantly tag important people and places without slowing anything down.
Flexibility: Because it learns the idea of an entity rather than a specific list, it works on almost any topic (politics, biology, manufacturing) without needing to be retrained.
The Future: This suggests that the "magic" of AI isn't just in the big, expensive models, but in the simple, structured patterns that emerge naturally when they learn to speak. We just need to learn how to listen to them.

In a nutshell: ToMMeR is a tiny, cheap, super-fast tool that unlocks the hidden ability of giant AI models to find names and things in text, proving that this skill is a natural part of how AI learns to understand the world.

1. Problem Statement

Entity Mention Detection is the foundational task of identifying text spans that refer to entities or concepts (e.g., "Marie Curie," "oxidation process"). Despite its critical role in Information Extraction (IE), it remains a performance bottleneck.

The Bottleneck: Current approaches often conflate mention detection with entity typing (classifying the entity type) in a single joint task. This obscures the specific learning of span boundaries.
The Limitation of Existing Methods:
- Traditional NER: Requires massive, task-specific datasets and hundreds of millions of parameters.
- Prompting LLMs: While flexible, prompting Large Language Models (LLMs) for extraction is computationally expensive, slow (auto-regressive generation), and requires schema specification at inference time.
- Gap: It is unclear whether LLMs inherently learn to detect entity boundaries during pretraining or if this capability must be explicitly taught via fine-tuning.

2. Methodology: ToMMeR

The authors propose ToMMeR (Token Matching for Mention Recognition), a lightweight probing architecture designed to extract mention detection capabilities from the early layers of frozen LLM backbones.

Core Architecture

Parameter Efficiency: The model adds <300K parameters (specifically ~274K in experiments) to a frozen LLM. It requires only a single partial forward pass (no text generation, no prompting).
Input: It takes token representations ( $z^\ell$ ) from a specific layer $\ell$ of a frozen LLM.
Mechanism: ToMMeR is based on the Binding ID framework from mechanistic interpretability, hypothesizing that LLMs implicitly bind related tokens (forming entity mentions) via learned signals in their hidden states.
Scoring Components:
1. Matching Score ( $m_{ij}$ ): Computes the association between token $i$ and token $j$ using a learned projection into a query-key subspace, normalized via cosine similarity (replacing softmax). This captures inter-token binding strength.
2. Token Values ( $v_i$ ): A linear probe on individual token representations, leveraging the observation that entity information is often concentrated in the final token of a span.
3. Span Probability: A logistic model combines the matching score, the max/min pooling of intermediate matching scores within the span, and the values of the span's last token and its immediate context to predict the probability that a span $(i, j)$ is a valid mention.

Training Strategy

Data: Trained on Pile-NER, a dataset of ~46k samples from "The Pile" annotated by GPT-3.5.
Loss Function: Uses Balanced Binary Cross-Entropy (BBCE) to handle the severe class imbalance between non-entity spans and entity spans.
Distillation: A two-stage training strategy is employed. After an initial fit, the model is used to augment the training data with unannotated mentions (reducing false negatives), effectively distilling the LLM's internal notion of entities.

3. Key Contributions

Efficient Probing Architecture: A simple, schema-agnostic model that recovers mention spans from a single forward pass of early LLM layers, achieving 42x faster inference than prompting methods.
Evidence of Emergent Capabilities: Demonstrates that diverse LLM architectures (ranging from 14M to 15B parameters, including autoregressive and encoder-only models) converge on highly similar mention boundaries (DICE score > 0.75). This suggests mention detection is a natural byproduct of language modeling objectives rather than a dataset artifact.
High-Performance Zero-Shot Detection: Achieves 93% recall across 13 diverse NER benchmarks without fine-tuning on the target task, while maintaining an estimated 90% precision (validated via human-calibrated LLM judges).
Modular Extension to Full NER: When extended with a lightweight span classification head, ToMMeR achieves competitive full NER performance (80–87% F1) on standard benchmarks, proving its utility as a generalist extraction framework.

4. Experimental Results

Zero-Shot Mention Detection

Recall: ToMMeR achieves an average 92.6% recall across 13 English benchmarks (e.g., CoNLL-2003, MultiNERD, OntoNotes).
Precision Challenge: Standard benchmarks (Gold) show low precision (~23%) because they use narrow, schema-specific definitions (e.g., missing nested entities or generic mentions).
LLM-Judged Precision: Using an LLM-as-a-Judge (GPT-4.1-mini) calibrated with human validation (Prediction-Powered Inference framework), the authors estimate the true precision is ~90%. This confirms that low gold precision is due to benchmark incompleteness, not model hallucination.
Cross-Lingual: Strong transfer to Latin-script languages (French, Spanish, German) with ~86% recall. Performance drops on Chinese due to tokenization differences in English-centric backbones.

Cross-Model Analysis

Convergence: Models ranging from 14M to 15B parameters (Pythia, LLaMA, Mistral, Phi, BERT) produce highly overlapping mention sets (DICE > 0.75).
Layer Analysis: Mention detection signals emerge very early (Layer 0-1) and remain stable through intermediate layers. Performance drops slightly at the final layer, likely because the model prioritizes next-token prediction over structural representation.

Full NER Performance

By adding a classification head to the detected spans, ToMMeR achieves 84.8% F1 on CoNLL-2003 and 80.4% F1 on OntoNotes using a LLaMA-3.2-1B backbone. This rivals or exceeds specialized models like EMBER and GLiNER, despite using a much smaller parameter footprint for the detection head.

5. Significance and Implications

Theoretical Insight: The work provides strong evidence that structured entity representations exist naturally in early transformer layers. Mention detection is not merely a downstream task but an emergent property of next-token prediction.
Practical Efficiency: ToMMeR offers a streaming-capable, low-latency solution for IE. It eliminates the need for expensive prompt engineering or full model fine-tuning, making it ideal for real-time RAG (Retrieval-Augmented Generation) pipelines and resource-constrained environments.
Schema Agnosticism: By decoupling detection from typing, ToMMeR enables flexible extraction pipelines where mentions can be detected first and typed later against any arbitrary schema, solving the "conflation" problem in traditional NER.

Limitations

Ground Truth Ambiguity: Evaluating untyped mention detection is difficult due to the lack of universal ground truth; the authors rely on LLM judges which, while validated, introduce some bias.
Architecture Constraints: The current implementation focuses on continuous spans (excluding discontinuous entities) and relies on autoregressive backbones which lack right-side context, potentially missing boundaries that bidirectional encoders (like BERT) capture more easily.
Training Data Bias: The model inherits biases from the GPT-3.5 annotations used in Pile-NER.

In conclusion, ToMMeR demonstrates that high-coverage, efficient entity mention detection is achievable by simply "probing" the latent representations of existing LLMs, challenging the necessity of massive, task-specific training for this fundamental NLP task.