Technical Summary: Evidence-Informed LLM Beliefs for Continual Scientific Discovery

Problem Statement

Open-ended scientific discovery using Large Language Models (LLMs) often operates as a long-horizon loop of hypothesis search and verification, guided by a reward signal. A prominent example is AutoDiscovery, which utilizes "Bayesian surprise"—the shift in an LLM's belief after observing evidence for a hypothesis—as a metric for discovery and a reward for search.

The authors identify a critical failure mode in this approach: static surprisal. AutoDiscovery treats the LLM's prior beliefs as static and unconditional, evaluating each new hypothesis in isolation against the model's parametric knowledge. This creates a mismatch with human scientific reasoning, where beliefs are non-stationary and evolve with experience. Consequently, static surprisal:

1. Overestimates progress: It counts derivative hypotheses (those implied by previous discoveries) as novel and surprising.
2. Wastes search budget: It repeatedly steers the agent toward saturated regions of the hypothesis space, leading to redundant "rediscoveries."

Human expert studies in neuroscience and social science confirmed this misalignment, revealing that approximately 29.54% and 25% of hypotheses deemed surprising by AutoDiscovery were substantially implied by prior discoveries in their respective domains.

Methodology

The paper proposes a framework for Evidence-Informed LLM Beliefs to enable non-stationary surprisal. The methodology consists of two main components: updating belief priors and modifying the search procedure.

1. Non-Stationary Surprisal via In-Context Belief Updating

Instead of using a static prior $P(\theta_H)$, the system computes a non-stationary prior conditioned on the history of previous hypotheses and their verification outcomes: $P(\theta_H | \{(H^{(i)}, V^{(i)}_D)\}_{i=1}^{t-1})$.

To implement this, the authors evaluated various In-Context Learning (ICL) mechanisms for retrieving and integrating past evidence:

• Full Context ($C_{full}$): Injecting all previous discoveries.
• Top-k Retrieval ($C_{top-k}$): Retrieving the $k$ most semantically similar previous hypotheses using embeddings.
• Structured Memory ($C_{mem0}$): Using a graph-based memory system (Mem0).

Findings: Embedding-based Top-k Retrieval (RAG) with $k=25$ proved most effective. It best anticipated the eventual posterior beliefs, reducing the total count of static surprisals by 37.43% on average compared to the static baseline. This method balances performance with computational cost (token usage) better than full context or complex memory structures.

2. Evidence-Informed Search

Simply replacing the static reward with non-stationary surprisal was found insufficient to redirect the search trajectory; the agent still explored similar hypotheses. To address this, the authors introduced two complementary mechanisms to the hypothesis selection step in the search algorithm (MCTS with Progressive Widening):

• Belief-Update Filtering: Before selecting a hypothesis for verification, the system checks if the hypothesis is already well-explained by prior evidence. It constructs an evidence-informed context and elicits an updated prior. If the absolute shift between the static prior and the evidence-informed prior exceeds a threshold ($\kappa = 0.2$), the hypothesis is filtered out. This prevents the agent from wasting budget on hypotheses that are no longer surprising under the updated belief state.
• Diversity Maximization: The system explicitly prioritizes exploration by selecting candidates with the lowest semantic similarity to previously verified hypotheses. This is calculated using the average cosine distance to the top-$k$ nearest previous hypotheses. This mechanism ensures the search explores new semantic regions rather than just avoiding immediate duplicates.
• Online De-duplication: An additional layer using hierarchical agglomerative clustering (HAC) removes candidates that are semantically redundant with already verified hypotheses or other candidates in the current node, preventing near-duplicate exploration.

Key Results

The proposed method was evaluated across five discovery domains (NLS-SES, Archaeology, Evolution Freshwater Fish, Fertility, Mortgage) using a search budget of 500 experiments per run.

• Accumulated Non-Stationary Surprisal: The full "Evidence-Informed Search" (combining belief-update filtering, diversity maximization, and online de-duplication) increased the accumulated non-stationary surprisal by 30.62% on average compared to the original static search procedure.
• Comparison to Baselines:
  • The method outperformed standard AutoDiscovery search by ~41 additional surprisals (30.63% gain).
  • It outperformed standard search with only online de-duplication by ~21 additional surprisals (13.37% gain), demonstrating that belief filtering and diversity maximization provide value beyond simple deduplication.
  • Simply replacing the reward with non-stationary surprisal without changing the selection mechanism yielded no significant improvement, highlighting the necessity of explicit search modifications.
• Semantic Diversity: The proposed method generated hypotheses with significantly lower average pairwise cosine similarity (0.448–0.452) compared to standard search (0.523–0.527), indicating broader semantic coverage.
• Ablation Studies: Removing any component (diversity maximization, belief-update filtering, or online de-duplication) degraded performance, confirming that each mechanism addresses a distinct source of redundancy.

Significance and Claims

The paper claims that continual scientific discovery with LLMs requires not only better belief measurement but also search procedures that actively manage redundancy and encourage diversity.

Key contributions include:

1. Identification of a Failure Mode: Demonstrating that static surprisal in continual discovery leads to the spurious rewarding of derivative hypotheses, validated by human experts.
2. Non-Stationary Surprisal: Introducing a method to update LLM priors with evidence from previous hypotheses, showing that embedding-based RAG best approximates the resulting posterior beliefs.
3. Evidence-Informed Search: Proposing a search strategy that combines belief-update filtering and diversity maximization, which successfully reallocates search budget away from saturated regions toward genuinely novel hypotheses.

The authors conclude that for long-horizon LLM discovery systems to be effective, they must continually revise what counts as "surprising" based on the cumulative epistemic state of the agent, rather than relying on a static model of the world.

Limitations

The authors acknowledge several limitations:

• Elicitation Quality: Belief estimates depend on prompting choices, sampling variance, and the LLM's ability to express uncertainty.
• Suppression of Valid Revisits: The non-stationary objective might suppress hypotheses that are legitimately worth revisiting under stronger or conflicting evidence.
• Scalability: The current experiments focus on controlled settings with finite budgets; scaling to open-ended workflows requires better verification mechanisms and long-term memory solutions beyond the current context window.