💻 computer science

Do not be greedy, Think Twice: Sampling and Selection for Document-level Information Extraction

The paper proposes "ThinkTwice," a framework that improves document-level information extraction by leveraging sampling to generate multiple candidate outputs and selecting the best one via unsupervised agreement or supervised reward models, thereby outperforming traditional greedy decoding methods.

Original authors: Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle, Eneko Agirre

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mikel Zubillaga, Oscar Sainz, Oier Lopez de Lacalle, Eneko Agirre

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex puzzle based on a long, messy story. Your goal is to pull out specific facts (like who did what, where, and when) and organize them into a neat, structured report. This is what the paper calls Document-level Information Extraction.

For a long time, when computers (specifically Large Language Models or LLMs) tried to do this, they used a method called Greedy Decoding. Think of this like a student taking a test who, at every single step, picks the one answer that looks most likely to be right immediately, without looking ahead or considering other possibilities. They just rush to the finish line with the first path that seems clear.

The authors of this paper, Mikel Zubillaga and his team, argue that this "rush to the finish" approach is actually holding the computer back. They propose a new framework called THINKTWICE.

Here is how THINKTWICE works, broken down into simple concepts:

1. The "Think Twice" Strategy (Sampling)

Instead of asking the computer to give just one answer, THINKTWICE asks it to generate many different versions of the report (the paper uses 64 different attempts).

The Analogy: Imagine you are a chef trying to make the perfect soup.
- Greedy Decoding is like tasting the spoonful you just made and immediately deciding, "This is the final recipe," without ever trying to adjust the salt or add more herbs.
- THINKTWICE is like cooking 64 slightly different versions of the soup. One might be too salty, one might be too spicy, but one of them might be perfect.

2. The "Judge" (Selection)

Once the computer has generated 64 different reports, you need a way to pick the best one. The paper tests two ways to be the "Judge":

The Unsupervised Judge (F1 Voting): This judge looks at all 64 reports and asks, "Which one agrees most with the others?" If 50 out of 64 reports say the suspect was "John," and only 10 say "Jane," the judge picks the version with "John." It's like a crowd-sourced decision where the most common, consistent answer wins.
The Supervised Judge (Reward Model): This is a smarter judge that has been trained on examples of "good" vs. "bad" reports. It learns to spot the subtle details that make a report high-quality, even if it's not the most common answer.

3. The "Reasoning" Boost

The paper also discovered that using models designed to "think" (Reasoning Models) makes a huge difference.

The Analogy: A standard model is like a fast typist who just types the first thing that comes to mind. A Reasoning Model is like a detective who stops to think, "Wait, if the bomb went off at 5 PM, the suspect couldn't have been at the bank at 4 PM."
The paper found that these "detective" models produce much better raw material for the "THINKTWICE" system to work with, especially for complex tasks.

4. Solving the "No Answer Key" Problem

To train the "Supervised Judge," you usually need an answer key (gold standard data) that includes the thought process (reasoning traces) the computer should have used. But for this specific type of task, those answer keys didn't exist.

The Solution: The authors used a clever trick called Rejection Sampling. They asked the computer to generate many attempts, kept the best ones, and used those "good" attempts as a fake answer key to teach the computer how to think better. It's like a student grading their own practice tests, keeping only the ones they got right, and using those to study for the real exam.

What Did They Find?

Thinking is better than rushing: Using the "Think Twice" method (generating many options and picking the best) consistently beat the standard "Greedy" method (picking the first likely answer).
Reasoning matters: Models that are designed to reason (think step-by-step) performed significantly better than standard models.
The "Judge" helps: Both the simple "crowd vote" judge and the trained "reward" judge improved the results. The trained judge was the best, setting a new record for how well computers can extract information from documents.
It works across languages: Even when they trained the system only on English data, it could still perform well on other languages (like Arabic or Chinese) when using this method, outperforming systems trained specifically on those languages.

In short: The paper shows that if you want a computer to extract facts from a long document, you shouldn't just ask it for one answer. You should ask it to brainstorm 64 different answers, and then use a smart system to pick the best one. This simple change, combined with "thinking" models, leads to much more accurate results.

Technical Summary: THINKTWICE for Document-level Information Extraction

Problem Statement

Document-level Information Extraction (DocIE) aims to convert natural language documents into structured templates containing entities, relations, and events. While Large Language Models (LLMs) have shown promise in this domain, standard practices rely on greedy decoding to ensure reproducibility and avoid output variability. However, this approach often fails to produce the optimal solution, as the most probable token at each step does not necessarily lead to the best global output. This limitation is particularly acute for reasoning-oriented models, where the gap between the greedy output and the best possible sampled output (the "oracle" score) is significantly wider. Furthermore, the field lacks high-quality "golden" reasoning trajectories required to fine-tune reasoning models for DocIE, and existing Test-Time Scaling (TTS) methods (like Majority Voting) are often ill-suited for the complex, structured outputs required in IE.

Methodology: THINKTWICE

The authors propose THINKTWICE, a sampling and selection framework designed to leverage the inherent variability of decoder-only LLMs to find higher-quality solutions. The framework operates in two main stages:

Sampling (Generation): Instead of generating a single output, the LLM is prompted to generate $N$ candidate templates (e.g., $N=64$ ) for a given document and set of annotation guidelines.
- For reasoning models, the process involves generating reasoning traces followed by the structured output.
- Constrained Decoding: To ensure validity, the generation is constrained to follow a predefined JSON schema.
- Silver Data Generation: To address the scarcity of golden reasoning traces, the authors employ a rejection-sampling method. They generate multiple reasoning-template pairs, score them against gold annotations using the dataset's evaluation metric, and retain the top $K$ pairs. These high-quality "silver" pairs are used to fine-tune the reasoning model.
Selection: A dedicated module selects the most suitable template from the $N$ candidates. The paper introduces two distinct selection strategies:
- Unsupervised Selection (F1 Voting): An alternative to Majority Voting. Instead of selecting the most frequent output (which may be empty or incorrect in complex IE tasks), this method calculates the pairwise F1-score similarity between all candidates. The candidate with the highest aggregate similarity score to the rest of the set is selected.
- Supervised Selection (Reward Model): A reward model is trained on a preference dataset constructed from the silver data. This dataset includes pairs of "chosen" (high-scoring) and "rejected" (low-scoring) templates. The model is trained using a Bradley-Terry loss function with a margin to learn to rank candidates, subsequently selecting the one with the highest predicted reward score.

Key Contributions

THINKTWICE Framework: A novel architecture that decouples generation and selection, demonstrating that sampling multiple candidates followed by a selection strategy outperforms standard greedy decoding in DocIE.
Novel Selection Strategies:
- The introduction of F1 Voting, an unsupervised selector tailored for IE that outperforms standard Majority Voting by utilizing continuous similarity metrics rather than binary matching.
- A supervised reward-based selector trained on automatically generated silver preference data, establishing new state-of-the-art results.
Rejection-Sampling for Silver Data: A method to generate high-quality reasoning traces for training reasoning models in the absence of golden trajectories, enabling effective supervised fine-tuning.
Empirical Analysis of Reasoning Models: A comprehensive study showing that reasoning-oriented LLMs consistently outperform non-reasoning counterparts in DocIE, especially when paired with sampling and selection.

Experimental Results

The authors evaluated THINKTWICE on MUC-4, MultiMUC (multilingual), and BETTER Granular datasets using Llama 3.3 and Qwen3 model families.

Reasoning vs. Non-Reasoning: Reasoning models consistently outperformed non-reasoning models in zero-shot settings, with the performance gap widening on complex datasets like BETTER.
Zero-Shot Performance: THINKTWICE with F1 Voting consistently outperformed greedy decoding baselines. On the BETTER dataset, F1 Voting improved Qwen3's performance by over 3 F1 points compared to greedy decoding.
Supervised Performance: When combined with supervised fine-tuning (using the rejection-sampled data) and the supervised reward selector, THINKTWICE achieved new state-of-the-art results on MUC-4, significantly surpassing previous encoder-based methods (e.g., ITERX) and greedy baselines.
Cross-Lingual Transfer: Models fine-tuned exclusively on English data, when paired with the supervised selector, successfully transferred to other languages (Arabic, Farsi, Korean, Russian, Chinese), outperforming systems trained directly on target-language data.
Oracle Gap: While THINKTWICE significantly narrows the gap to the oracle (the best possible sampled output), a substantial margin remains, indicating room for further improvement in selection strategies.

Significance and Claims

The paper claims that greedy decoding fails to fully exploit the capabilities of modern LLMs for Document-level Information Extraction. By treating output variability not as a limitation but as a resource, THINKTWICE demonstrates that Test-Time Scaling (TTS) techniques can be effectively adapted to IE tasks.

The authors emphasize that:

Simply scaling model size or using greedy decoding is insufficient to unlock the full potential of reasoning models.
Effective test-time scaling strategies (sampling + selection) are critical for achieving state-of-the-art performance.
High-quality supervision for reasoning models can be synthesized via rejection sampling, overcoming data scarcity.
The proposed framework achieves superior results in both monolingual and cross-lingual settings, outperforming proprietary models (like GPT-5.5 in the paper's context) and previous supervised state-of-the-art methods.

The work concludes that while computational overhead increases due to multiple generations, the trade-off yields substantial performance gains, and the remaining gap to oracle performance suggests a promising direction for future research in IE.