💬 NLP

MUSE: A Full-Text Cross-Domain Knowledge Base of Scientific Problems, Solutions, and Rationales

This paper introduces MUSE, a full-text, cross-domain knowledge base containing 37,000 expert-curated Problem-Solution-Rationale triplets derived from scientific literature, which demonstrates that training large language models with rationale supervision enhances performance on complex, multi-constraint problems while potentially hindering it on simpler ones.

Original authors: Tsofia Cohen, Tom Hope

Published 2026-08-12

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Tsofia Cohen, Tom Hope

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the world of science as a massive, endless library where every book is a research paper. For a long time, computers trying to read these books have been like students who only skim the table of contents or the very first page. They can tell you the title of the book or the main character, but they often miss the messy, brilliant, and specific details of how the characters solved their problems. Scientists don't just state a problem and a solution; they write long, winding stories explaining why they chose that specific solution over another. This is the "why" behind the "what." If we want computers to truly understand science—like a smart apprentice rather than a basic librarian—we need to teach them to read these hidden stories of reasoning. This is where a new project called MUSE comes in, aiming to turn those hidden stories into a structured map that anyone, or any computer, can follow.

The paper introduces MUSE (Mining Underlying Scientific Explanations), a giant new digital treasure chest filled with 36,960 carefully extracted "Problem–Solution–Rationale" triplets. Think of it like a recipe book, but instead of just listing ingredients (the solution) and the final dish (the result), MUSE captures the chef's secret notes: the specific problem they were trying to fix (like "the sauce is too watery"), the exact fix they applied (adding a thickener), and the chef's reasoning for why that specific thickener was the best choice (because it wouldn't change the flavor). The researchers built this by first hand-annotating 579 paragraphs from scientific papers to create a "gold standard" guide. Then, they trained a team of AI tools to act like a high-tech mining crew, scanning millions of full-text papers across many different fields to find these specific three-part stories.

The team found that while a single, all-powerful AI model tried to read a whole paragraph and guess the problem, solution, and reason all at once, it often got confused, mixing up the "why" with the "how." So, they built a modular pipeline instead: a step-by-step assembly line where one tool finds the problem, another finds the solution, and a third explains the connection. This approach worked much better, successfully turning raw text into a clean, organized database. When they tested this new knowledge base by teaching a large language model (an AI) to solve scientific problems using these "rationales" as a guide, they discovered something interesting. The AI got significantly better at solving complex problems that had many tricky constraints, suggesting that understanding the "why" helps when the path is foggy. However, for simple problems, adding the extra "why" information actually made the AI slightly worse, as if it was overthinking a task that didn't need it. This suggests that while MUSE is a powerful new tool for helping computers learn from expert reasoning, it's not a magic wand that fixes everything; it shines brightest when the scientific puzzle is truly difficult.

Technical Summary: MUSE – A Full-Text Cross-Domain Knowledge Base of Scientific Problems, Solutions, and Rationales

1. Problem Statement

Scientific Natural Language Processing (NLP) has advanced in extracting structured entities (e.g., datasets, methods) and discourse roles (e.g., background, conclusion) from scholarly text. However, existing resources often fail to capture a fundamental, fine-grained structure prevalent in scientific papers: the specific Problem–Solution–Rationale (P–S–R) triplet.

Current datasets typically operate at the abstract or paper level, focusing on high-level contributions. They often miss technical sub-problems buried deep within full-text sections (e.g., methods, limitations, appendices) where authors detail specific obstacles, the precise mechanisms used to overcome them, and the explicit reasoning justifying those choices. Furthermore, existing resources rarely link these elements with the necessary granularity to distinguish between a solution, the problem it addresses, and the author's specific motivation for that choice. This gap limits the ability to perform cross-domain ideation, evaluate scientific reasoning, or train models to understand the "why" behind scientific decisions.

2. Methodology

The authors introduce MUSE (Mining Underlying Scientific Explanations), a resource comprising a curated seed dataset and a large-scale knowledge base (KB) constructed via a modular extraction pipeline.

A. The P–S–R Task and Annotation Schema
The task is defined as recovering P–S–R structures from scientific paragraphs:

Problem: A concrete limitation, obstacle, gap, or desideratum.
Solution: A method, mechanism, design choice, or procedure used to address the problem.
Rationale: The author-stated motivation, justification, or explanation connecting the solution to the problem.

The annotation schema captures:

Salient Spans: Minimal text spans for problems, solutions, and rationales grounded in the source.
Relations: solves links (solution to problem) and rationale_of links (rationale to solution).
Conceptual Coreference: Grouping fragmented or repeated mentions of the same underlying concept.

B. The Extraction Pipeline
To avoid the brittleness of end-to-end Large Language Model (LLM) extraction—which the authors found often conflates solutions with rationales or fails to preserve correspondence in complex paragraphs—they designed a modular pipeline:

Paragraph Filtering: A cascaded classifier first identifies paragraphs containing Problem–Solution pairs, then checks for the presence of a rationale. For KB construction, only the top 10% of paragraphs ranked by confidence are retained to prioritize precision.
Span Extraction: Token-level classifiers (specifically multi-class DeBERTa-v3-large) extract problem_salient and solution_salient spans. Rationale spans are noted to be less boundary-stable and are not relied upon for token classification in the final output.
Relation Extraction: A fine-tuned Mistral-7B model handles conceptual coreference (grouping mentions) and the solves relation (linking solution groups to problem groups).
Post-Processing and Rationale Construction: An LLM (GPT-4o) is prompted with the source paragraph and extracted spans to generate self-contained, standalone descriptions for the problem and solution. A subsequent prompt extracts a faithful, source-grounded rationale. A final refinement step using Claude Opus ensures semantic separation between the solution (technical approach) and rationale (motivation/justification).

C. Dataset Construction

Seed Dataset: 579 expert-annotated full-text paragraphs from arXiv, curated by a PhD-level scientific annotator and an NLP expert, with high inter-annotator agreement.
Knowledge Base: The pipeline was applied to full-text arXiv papers across all domains (through 2026), resulting in 36,960 source-grounded P–S–R triplets.

3. Key Results

Extraction Performance:

Classification: The cascaded classifier approach significantly outperformed holistic classifiers, achieving an F1 of 0.73 on the relevant class for the top-confidence subset.
Span Extraction: Multi-class DeBERTa-v3-large achieved the best performance for problem and solution spans (F1 ~0.60–0.66), while rationale spans remained difficult to capture via token classification alone.
Relation Extraction: The fine-tuned Mistral-7B achieved strong performance on coreference (F1 = 0.91) and solves links (F1 = 0.87 for positive links).
Post-Processing: Human evaluation of the generated standalone fields showed high scores for standalone quality (Problem: 4.97/5; Solution: 4.82/5) and fidelity. Rationale construction scored well on groundedness (4.70) and coherence (4.84), though specificity and completeness were moderate.

Preliminary Application (Rationale-Supervised Tuning):
The authors conducted a preliminary experiment training LLMs (Qwen-3-32B and DeepSeek-R1) to solve scientific problems using MUSE data. They compared:

PS: Problem $\to$ Solution (baseline).
PRS: Problem $\to$ Rationale $\to$ Solution.

Findings:

Complex Problems: Rationale supervision improved performance on complex, multi-constraint problems (e.g., Qwen-3-32B's overall quality rose from 5.12 to 6.41).
Simple Problems: Rationale supervision could harm performance on trivial problems, suggesting that the additional reasoning step may introduce noise or over-complication for straightforward tasks.
Conclusion: The rationale field provides a useful training signal for complex reasoning but is not universally beneficial.

4. Contributions

The paper claims three primary contributions:

A Full-Text Scientific IE Task and Seed Dataset: A new expert-annotated dataset for P–S–R extraction covering full-text paragraphs, including salient spans, relational links, and conceptual coreference.
A Large Structured Knowledge Base: A collection of 36,960 P–S–R triplets mined from full-text arXiv papers, representing self-contained problem, solution, and rationale fields with source provenance.
A Preliminary Application Study: An investigation demonstrating that rationale-supervised fine-tuning can enhance LLM performance on complex scientific problem-solving tasks, while highlighting potential downsides for simpler tasks.

5. Significance and Claims

The authors position MUSE as a resource that bridges the gap between high-level paper summaries and the granular technical reasoning found in full texts. By explicitly modeling the rationale—the "why" behind a solution—MUSE aims to support:

AI Scientific Problem Solving: Providing training data for models to reason through constraints.
Cross-Domain Ideation: Enabling the discovery of analogous rationales across different scientific fields.
Meta-Scientific Analysis: Allowing researchers to study how different fields formulate constraints and justify methods.

The paper maintains a modest tone regarding its application study, presenting the LLM tuning results as a "preliminary experiment" to illustrate the utility of the resource rather than claiming a state-of-the-art reasoning model. The primary value proposition remains the creation of a high-quality, structured, and cross-domain knowledge base of scientific explanations that has not previously existed at this scale.