💬 NLP

Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty

This paper introduces Think-Probe-Respond (TPR), a lightweight method that probes latent novelty judgments from a model's hidden states to correct its systematic bias toward "medium novelty" ratings, thereby significantly improving the accuracy of large language models in evaluating research idea novelty.

Original authors: Tim Schopf, Tobias Schreieder, Akiko Aizawa

Published 2026-08-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tim Schopf, Tobias Schreieder, Akiko Aizawa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Science advances on the back of new ideas. Researchers constantly propose fresh ways to solve problems, but before an idea can change the world, it must be judged: is it truly new, or just a slight variation of what already exists? This judgment is usually done by human experts who read thousands of papers to find the subtle differences. It is slow, expensive, and hard to scale as the volume of scientific work grows. In recent years, scientists have turned to large language models—powerful computer systems trained on vast amounts of text—to help with this task. These models can read a research idea and a list of related past work, then write a paragraph explaining why the idea is new or old. They are surprisingly good at writing these explanations, often sounding just like a human expert.

However, a strange disconnect has emerged. While these computer systems write excellent explanations, their final verdict on how new an idea is often misses the mark. They tend to play it safe, labeling most ideas as "somewhat new" even when their own written reasoning suggests the idea is either completely old or groundbreakingly new. It is as if the system knows the truth but is afraid to say it, defaulting to a middle ground that feels safe but is inaccurate. This hesitation creates a bottleneck in automated science, where the tool that should speed up discovery ends up blurring the very distinctions it is meant to find.

A team of researchers from Japan and Germany set out to understand why this happens and how to fix it. They discovered that the problem lies in how the computer models process information. When a model is asked to judge a research idea, it goes through a hidden phase of thinking, generating a series of internal steps before producing a final answer. The researchers found that during this thinking phase, the model actually forms a clear, accurate belief about the novelty of the idea. This belief is encoded in the model's internal state, a complex web of numbers that represents what the system "knows" at that moment. The trouble arises when the model tries to translate this internal knowledge into a final number. Instead of reporting what it truly believes, it drifts toward the middle of the scale, ignoring the strong evidence it just generated.

To solve this, the team developed a method called Think-Probe-Respond. The process works in three simple stages. First, the model is asked to think step by step about a research idea, just as it normally would. During this thinking phase, the researchers pause the system and take a "snapshot" of its internal state. They use a small, simple tool to read this snapshot and extract the model's true, hidden judgment about the novelty of the idea. This is the "probe" step. The researchers then take this hidden judgment and feed it back into the model as a hint. Finally, the model is asked to respond, but this time it must write its final answer and justification while keeping that hidden judgment in mind. By forcing the model to align its final output with the belief it formed during thinking, the system stops drifting toward the safe middle ground.

The results were striking. When tested on a dataset of over 1,300 expert-annotated research ideas, the new method improved the accuracy of the models by more than 22 percent compared to standard approaches. The models became much better at identifying ideas that were either completely unoriginal or truly revolutionary, rather than lumping everything into the middle categories. Remarkably, this improvement did not require retraining the massive computer models, which would be incredibly expensive and time-consuming. The researchers only needed to train a tiny, simple classifier to read the internal states, a process that could be done quickly on standard computer hardware. Even smaller, less powerful models, when guided by this method, performed better than much larger models using standard techniques.

The study also revealed exactly when the model forms its best judgment. The researchers found that the internal signal for novelty is strongest right at the end of the thinking phase, just before the model starts writing its final response. If they tried to read the signal earlier, while the model was still thinking, the information was less clear. If they waited until the model started writing the final response, the signal became diluted as the system focused on choosing words and formatting the text. This suggests that the true "decision" happens in the quiet space of reasoning, before the system begins to speak.

This work offers a practical way to make artificial intelligence a more reliable partner in science. It shows that these models are not necessarily confused about what is new; they are simply biased toward caution when they have to give a final number. By listening to their internal reasoning before they speak, researchers can unlock a more accurate and honest assessment of scientific ideas. The method is lightweight and efficient, meaning it can be applied to many different types of models without needing massive computing power. While the study focused on machine learning research, the approach suggests a broader path forward: if we can learn to read the hidden thoughts of these systems, we can help them overcome their own biases and provide the clear, decisive judgments that science needs to move forward.

Technical Summary: Think-Probe-Respond (TPR) for Research Idea Novelty Judgment

Problem Statement

Automated novelty judgment is critical for accelerating scientific discovery, yet manual evaluation is time-consuming and difficult to scale. While Large Language Models (LLMs) are increasingly adopted to judge research idea novelty, a critical discrepancy persists: although LLMs generate reasoning rationales that closely mirror human experts, their final numerical novelty judgments often diverge significantly from human consensus.

The authors identify a specific failure mode: a systematic miscalibration where LLMs exhibit a strong bias toward conservative "medium novelty" judgments (classes 3 and 4), even when their internal reasoning supports substantially different conclusions (e.g., highly novel or not novel). This bias leads to poor performance on extreme novelty categories, resulting in low macro-F1 scores despite high alignment in textual justifications.

Methodology: Think-Probe-Respond (TPR)

To address this miscalibration, the authors propose Think-Probe-Respond (TPR), a lightweight, inference-time approach that decouples the model's latent beliefs from its final output generation. TPR operates in three distinct stages:

Think: The LLM is instructed to evaluate the novelty of a research idea based on related works and textual descriptions of novelty categories (without explicit numerical scores). The model is prompted to "think step by step" (or utilizes its native reasoning tokens) to generate a qualitative reasoning trace. Crucially, the model is prevented from generating a numerical score at this stage to avoid anchoring its internal representations to specific outcomes.
Probe: Upon the generation of the final "think" token ( $t_n$ ), the system extracts the hidden state representation ( $h^{(L)}_{t_n}$ ) from the model's final layer. This representation is fed into a simple, trainable logistic regression classifier (the probing classifier). This classifier predicts a latent novelty score (1–5) based on the model's internal state before the final response is generated.
Respond: The textual description of the predicted novelty class is appended to the model's output. The LLM then resumes generation, conditioned on both its prior reasoning and the probed novelty judgment, to produce the final justification and numerical score.

Key Technical Characteristics:

Parameter Efficiency: All LLM parameters remain frozen; only the lightweight logistic regression classifier is trained (on CPU).
Timing: Probing occurs specifically at the end of the reasoning phase ( $t_n$ ), as experiments show novelty signals are most consolidated at this point, whereas probing during response generation dilutes the signal.
Applicability: The method is designed for open-source LLMs where hidden states are accessible.

Experimental Setup

Benchmark: Experiments were conducted on RINoBench, the only publicly available benchmark for research idea novelty judgment with human-annotated scores (1,381 expert-authored ideas).
Baselines: The authors compared TPR against zero-shot prompting, few-shot prompting, Chain-of-Thought (CoT), various prompt-based agents (e.g., Moose, ResearchAgent), and a computationally expensive Fine-Tuning approach using LoRA.
Models: Evaluation included diverse open-source reasoning (e.g., Qwen3, GPT-OSS) and non-reasoning models (e.g., Gemma 3, Llama 3.1) across various parameter sizes (4B to 70B).

Key Results

Performance Improvement: Across all evaluated models, TPR achieved the highest performance, improving novelty judgment macro-F1 scores by an average of 22.30% over the best competing baselines.
Mitigation of Bias: TPR successfully mitigated the prevalent "medium novelty" bias. Unlike zero-shot prompting, which concentrated predictions on classes 3 and 4, TPR enabled models to correctly identify extreme novelty cases (classes 1 and 5), resulting in more balanced class-wise F1 scores.
Efficiency vs. Fine-Tuning: TPR significantly outperformed the Fine-Tune (LoRA) approach, demonstrating that probing latent beliefs is more effective than parameter-heavy fine-tuning for this specific task.
Model Agnosticism: The method proved robust across different model families and sizes. Notably, smaller models (e.g., Gemma-3-4B) using TPR outperformed substantially larger proprietary models (e.g., GPT-5, Claude Opus) used in zero-shot settings.
Reasoning vs. Non-Reasoning: TPR was effective for both reasoning and non-reasoning models, though non-reasoning models with explicit "think step-by-step" instructions showed the largest gains.

Significance and Claims

The paper claims that the miscalibration of LLMs as novelty judges is best understood as a mismatch between latent belief and expressed judgment. The models "know" the correct novelty level during their reasoning phase but fail to translate this into accurate final outputs due to a bias toward safe, middle-ground categories.

The significance of TPR lies in its ability to:

Unlock Latent Knowledge: Explicitly exploit the internal beliefs formed during reasoning to guide the final output, rather than relying solely on the generation process.
Provide a Lightweight Solution: Offer a computationally efficient alternative to fine-tuning that yields superior results without modifying the base model weights.
Improve Scientific Assessment: Enable more accurate and balanced automated novelty assessments, which are crucial for helping researchers refine ideas and for accelerating scientific discovery.

The authors emphasize that while TPR improves automated judgment, novelty remains inherently subjective. The approach is intended as a tool to support, not replace, human expert judgment.