MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning

1. Problem Statement

While Large Language Models (LLMs) have advanced in complex reasoning, Process-Level Reward Models (PRMs)—which evaluate intermediate reasoning steps rather than just final outcomes—remain under-evaluated in safety-critical domains like medicine.

• Gap in Existing Benchmarks: Current PRM benchmarks (e.g., PRMBench, ProcessBench) focus primarily on mathematics. They fail to capture the unique characteristics of medical reasoning: safety criticality (errors can cause patient death), knowledge intensity (requiring integration of pharmacology, guidelines, etc.), and inherent uncertainty (differential diagnosis).
• Consequences: Without a dedicated evaluation framework, it is impossible to quantify a model's ability to detect specific medical reasoning errors (e.g., missing drug contraindications or skipping prerequisite checks) before deployment, leaving real-world healthcare applications unverified and potentially unsafe.

2. Methodology

The authors propose MedPRMBench, the first process-level reward model benchmark for the medical domain. The construction involves a rigorous three-phase pipeline based on Clinical Reasoning Blueprints (CRBs).

A. Data Construction Pipeline

1. Phase 1: Data Curation:
   • Aggregated 7 medical QA sources (MedQA, MedMCQA, MMLU-Medical, etc.).
   • Applied difficulty-aware filtering to retain challenging questions (where models fail >50% of the time).
   • Generated verified step-by-step reasoning traces via rejection sampling using frontier models for datasets lacking expert reasoning text.
2. Phase 2: Clinical Reasoning Blueprint (CRB) Construction:
   • Extracted Evidence Reasoning Networks (ERNs) from reasoning text using multi-model semantic voting to ensure logical connectivity.
   • Distilled ERNs into CRBs: structured causal subgraphs representing the core path from evidence to conclusion.
   • Annotated CRBs with safety-criticality levels (Critical, Major, Moderate, Minor), node criticality scores, and prerequisite relationships.
3. Phase 3: Blueprint-Guided Error Injection:
   • Injected controlled errors into correct reasoning chains based on a 14-type error taxonomy.
   • Used the CRB structure to target specific nodes (e.g., high-criticality nodes for safety errors).
   • Employed deterministic text diff verification and multi-model voting to ensure label accuracy.
   • Conducted expert review where medical professionals validated errors and corrected reasoning, with a strict consensus mechanism (human + 3 models must agree) to filter out conflicts.

B. Error Taxonomy

The benchmark defines 14 fine-grained error types across three categories, extending beyond general reasoning to medical specifics:

• Simplicity (2 types): Non-Redundancy (S-1), Non-Circular Logic (S-2).
• Soundness (7 types): Evidence-Based Soundness (R-1), Step Consistency (R-2), Contextual Applicability (R-3), Confidence Invariance (R-4), Safety Awareness (R-5), Information Grounding Compliance (R-6), Trajectory Reasoning (R-7).
• Sensitivity (5 types): Prerequisite Sensitivity (E-1), Deception Resistance (E-2), Multi-Solution Consistency (E-3), Quantitative Correctness (E-4), Differential Diagnosis Coverage (E-5).
• Severity Grading: Introduced a 4-level severity system (Critical, Major, Moderate, Minor) to quantify clinical impact.

C. Dataset Statistics

• Total: 13,379 questions.
• Evaluation Set: 6,500 questions, 13,000 reasoning chains, 113,910 step-level labels.
• Training Set: 6,879 questions.
• Granularity: Average 9.6 steps per chain.

3. Key Contributions

1. First Medical PRM Benchmark: MedPRMBench is the inaugural benchmark specifically designed for evaluating process-level reward models in medical reasoning.
2. Novel Construction Methodology: Introduced a Blueprint-Guided Error Injection approach using Clinical Reasoning Blueprints (CRBs) to ensure errors are structurally grounded and clinically relevant, rather than random perturbations.
3. Specialized Taxonomy & Grading: Defined a 14-type error taxonomy tailored to medical reasoning (including Safety Awareness and Prerequisite Sensitivity) and the first 4-level severity grading system for quantifying clinical impact.
4. Comprehensive Evaluation: Provided a systematic evaluation of 21 models (open-source medical/reasoning models, proprietary frontier models, and medical PRMs), revealing critical weaknesses in current systems.

4. Results

The authors trained a medical PRM baseline (based on Qwen3-8B) and evaluated it against various baselines.

• Superior Performance: The proposed medical PRM achieved an overall PRMScore of 87.1%, significantly outperforming the best proprietary critic model (GPT-5.4 at 75.4%) and existing medical PRMs (MedSSS_Policy at 54.5%).
• Error Detection Capabilities:
  • The PRM excelled in detecting Simplicity errors (where other models struggled) and medical-specific types like Safety Awareness (86.9%) and Differential Diagnosis Coverage (85.7%).
  • It demonstrated the most balanced bias profile (Bias Gap of +4.0), avoiding the extreme positive bias (ignoring errors) of open-source models or the negative bias (flagging everything as errors) of some proprietary models.
• Hard Subset Stress Test: On a "Hard Subset" of 700 subtle errors (where errors do not change the final answer), the PRM was the only model to improve performance (+0.9 pp), while all other models degraded.
• Downstream Utility: When used as a plug-and-play verifier for medical QA, the PRM improved downstream accuracy by 3.2% to 6.7% across different policy models, demonstrating its practical value in filtering erroneous reasoning chains.

5. Significance

• Safety Standardization: MedPRMBench provides a standardized tool to assess the safety and reliability of medical AI, addressing the critical need to detect reasoning errors that could lead to patient harm.
• Guidance for Future Research: The benchmark reveals that current models lack robustness in detecting subtle, safety-critical errors and structural reasoning flaws, providing clear directions for improving medical PRMs.
• Clinical Applicability: By proving that process-level verification can significantly boost medical QA accuracy, the work validates the integration of PRMs into clinical decision support systems to enhance safety and trustworthiness.

In conclusion, MedPRMBench fills a critical gap in medical AI evaluation, moving beyond outcome-based metrics to ensure that the reasoning process itself is safe, logical, and clinically sound.