💬 NLP

Just Pass Twice: Efficient Token Classification with LLMs for Zero-Shot NER

The paper proposes "Just Pass Twice" (JPT), a method that enables causal large language models to perform efficient, state-of-the-art zero-shot Named Entity Recognition by concatenating input text to allow full bidirectional context without architectural changes, thereby overcoming the limitations of autoregressive generation while achieving over 20x faster inference.

Original authors: Ahmed Ewais, Ahmed Hashish, Amr Ali

Published 2026-08-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ahmed Ewais, Ahmed Hashish, Amr Ali

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Language is a web of context, where the meaning of a single word often depends entirely on the words that come after it. Consider the word "Paris." If you encounter it at the start of a sentence, your brain does not yet know if it refers to the French capital or the musician who released a new album. To resolve this ambiguity, you must read the entire sentence. For decades, computers struggled with this same problem. Traditional tools for finding names of people, places, and organizations in text were built to look backward, analyzing words one by one as they appeared. They could not see the future, so they often guessed wrong when the crucial clue appeared later in the sentence. Meanwhile, the newest, most powerful computer language models could understand vast amounts of human knowledge, but they were built to write text, not to label it. They, too, were forced to look only backward, making them slow and prone to errors when asked to identify specific words in a block of text.

A team of researchers at WitnessAI has found a way to give these powerful models the ability to see the whole picture without changing their underlying design. They call their method "Just Pass Twice." The solution is deceptively simple: instead of feeding a sentence to the computer model just once, they feed it twice, back-to-back. The first time, the model reads the sentence normally. The second time, it reads the exact same sentence again. Because the model is designed to remember everything it has seen so far, the second reading allows every single word to "look back" at the entire first reading. This means that when the model analyzes the word "Paris" during the second pass, it can already see the words "new album" that appeared later in the first pass. This trick effectively gives the model a view of the entire sentence at once, allowing it to make precise decisions about what each word means, all while running at a speed that is more than twenty times faster than the previous best methods.

The researchers tested this approach on a wide variety of tasks where computers must identify specific types of information, such as names of people, organizations, and locations, across different fields like music, politics, and science. They found that by combining this "double reading" technique with clear, written definitions of what each type of entity is, the model became incredibly accurate. In tests, it outperformed the best existing methods by a significant margin, improving its accuracy by nearly eight points on average. This is a substantial leap in performance for a task that has been studied for decades. The model did not just guess better; it understood the context more deeply. For instance, it could distinguish between a "person" and a "politician" or a "country" and an "organization" by relying on the specific definitions provided, rather than just memorizing a list of names.

What makes this discovery particularly striking is how it bypasses the usual trade-offs in artificial intelligence. Typically, making a model more accurate requires making it slower or more complex. Generative models, which create text word by word, are slow because they must wait for each word to be written before moving to the next. Discriminative models, which simply label words, are fast but often lack the deep understanding of the newest, largest models. The "Just Pass Twice" method bridges this gap. It allows a large, knowledge-rich model to perform the fast, precise job of labeling text. The researchers achieved this without altering the model's architecture or requiring it to be retrained from scratch. They simply changed the input, duplicating the text so the model could process it in a single, parallel burst of calculation rather than a slow, sequential one.

The implications of this work extend beyond just finding names in text. It demonstrates that the limitations of modern language models are not always due to a lack of intelligence, but sometimes due to the way they are asked to work. By finding a way to let these models see the full context of a sentence, the researchers have unlocked a new level of efficiency and accuracy. Their method is not a magic trick or a complex algorithmic overhaul; it is a straightforward adjustment that leverages the model's existing capabilities in a new way. The result is a system that is both faster and smarter, capable of handling the nuances of human language with a level of precision that was previously out of reach for this class of tools. This approach suggests that the path forward for artificial intelligence may not always require building bigger or more complex machines, but rather finding simpler, more elegant ways to use the ones we already have.

Technical Summary: Just Pass Twice (JPT)

Problem Statement

Named Entity Recognition (NER) is a foundational task for information extraction, typically framed as token-level sequence labeling. While discriminative models based on bidirectional encoders (e.g., BERT, DeBERTa) excel at this task by leveraging full context, they are limited by their parameter size and world knowledge, hindering zero-shot generalization to novel domains or entity types. Conversely, Large Language Models (LLMs) possess vast world knowledge and reasoning capabilities but employ causal (unidirectional) attention. This architectural constraint prevents tokens from attending to future context, which is often essential for disambiguating entity types (e.g., distinguishing "Paris" as a person vs. a city requires seeing "album" later in the sentence).

Existing LLM-based NER approaches attempt to bypass this by using generative formulations (prompting the model to list entities). However, these methods suffer from three critical drawbacks:

Speed: Autoregressive decoding is inherently sequential and slow compared to single-pass classification.
Cost: Generating output tokens is significantly more expensive than processing input tokens in commercial APIs.
Reliability: Generative models are prone to hallucinating entities not present in the input and producing formatting errors.

Methodology

The authors propose Just Pass Twice (JPT), a method that enables causal LLMs to perform discriminative token classification with full bidirectional context without modifying the underlying architecture.

1. Bidirectional Context via Input Duplication

The core insight of JPT is that duplicating the input sequence allows tokens in the second pass to attend to the complete sentence.

Mechanism: The input sequence $x$ is concatenated with itself: $x' = (x, [SEP], x)$ .
Attention Flow: When the causal LLM processes the second pass, a token at position $k$ can attend to all preceding tokens in the second pass (standard past context) and, crucially, all tokens in the first pass (future context relative to the original sentence).
Efficiency: This operation occurs entirely during the prefill phase, which is highly parallel and compute-bound. Unlike autoregressive decoding, which is memory-bound and sequential, JPT processes the doubled input in a single forward pass, avoiding the latency of token-by-token generation.

2. Definition-Guided Entity Typing

To achieve flexible zero-shot generalization, JPT moves beyond simple label names (e.g., "PERSON") and utilizes natural language definitions.

Dual-Channel Injection: Entity definitions are injected via two channels:
1. Embeddings: Definitions are encoded by a text encoder and projected into a shared space to serve as entity embeddings.
2. Prompt: Definitions are included directly in the LLM's input prompt to guide token encoding via attention.
Control: This allows users to specify boundary cases and disambiguation rules in natural language (e.g., defining whether "nearby" counts as a location), decoupling the model from fixed vocabularies.

3. Architecture and Training

Backbone: JPT uses frozen decoder-only LLMs (Qwen3-4B and Qwen3-8B).
Adapters: Only lightweight components are trained: LoRA adapters on attention projections, token/entity projection MLPs, and a bilinear classifier. This updates less than 2% of the backbone parameters.
Classification: A bilinear classifier computes matching scores between projected token representations (from the second pass) and entity embeddings. An explicit "O" (outside) class is included.
Data: The model is trained on a proprietary Wikipedia-derived dataset with no overlap with evaluation benchmarks, ensuring true zero-shot evaluation.

Key Contributions

Architectural Innovation: A simple input transformation (duplication) that enables bidirectional token classification in causal LLMs without architectural changes or special pretraining.
Definition-Guided Typing: A mechanism for flexible zero-shot generalization using natural language definitions injected via both embeddings and prompts, offering fine-grained control over entity boundaries.
Performance and Efficiency: The method achieves state-of-the-art results while being significantly faster than generative alternatives.

Experimental Results

The authors evaluated JPT on CrossNER (five specialized domains) and MIT (Movie/Restaurant) benchmarks, as well as an extended suite of 20 diverse datasets.

Accuracy: JPT-8B achieves a state-of-the-art F1 score, surpassing the previous best method (SaM) by +7.9 F1 on average across CrossNER and MIT. It outperforms UniNER-7B by +12.3 F1.
Generalization: JPT-4B outperforms both GLiNER-L and UniNER-7B on 19 out of 20 extended benchmarks, demonstrating robust generalization across biomedical, social media, and multilingual domains.
Efficiency: JPT is over 20× faster than comparable generative methods. For instance, JPT-4B is approximately 22× faster than UniNER-7B. This is attributed to the parallel prefill nature of the method versus the sequential decoding of generative models.
Ablation Studies:
- Removing input duplication (single pass) degrades performance by -15.2 F1, confirming the necessity of bidirectional context.
- Using definitions in only one channel (prompt or embedding) yields limited gains; the dual-channel approach provides the highest performance (+12.6 F1 over no definitions).

Significance and Claims

The paper claims that JPT demonstrates that causal attention constraints do not necessitate generative approaches for token-level tasks. By leveraging the massive parameter space and world knowledge of decoder-only LLMs while bypassing autoregressive generation, JPT offers a solution that is both highly accurate and computationally efficient.

The authors emphasize that their approach:

Eliminates the latency and hallucination issues inherent in generative NER.
Leverages the reasoning capabilities of large models without the cost of sequential decoding.
Provides a controllable information extraction framework where entity boundaries are defined by natural language specifications rather than fixed labels.

The work suggests that simple input transformations can overcome architectural bottlenecks, potentially extending to other token-level tasks beyond NER. The authors note limitations regarding sequence length (doubling input increases $O(N^2)$ complexity) and current support for flat NER only (nested entities require post-hoc merging), but assert that the method's efficiency and accuracy make it a strong candidate for real-world deployment.