Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyond

Technical Summary: Evaluating ChatGPT on Medical Information Extraction Tasks

1. Problem Statement

While Large Language Models (LLMs) like ChatGPT have demonstrated remarkable general-purpose natural language processing (NLP) capabilities, their efficacy in specialized domains—specifically Medical Information Extraction (MedIE)—remains a critical area of investigation. MedIE is a highly complex field requiring deep domain knowledge to extract structured factual knowledge (entities, relations, and events) from unstructured clinical texts. Existing research often focuses on general NLP tasks or uses small, random subsets of benchmarks, which may not accurately reflect the model's performance in high-stakes, specialized medical scenarios.

2. Methodology

The authors conducted a systematic, multi-dimensional evaluation of ChatGPT (specifically the gpt-3.5-turbo engine) across four fine-grained MedIE tasks using six benchmark datasets:

• Named Entity Recognition (NER): Identifying and classifying medical entities (e.g., ShARe13, CADEC, CMeEE-v2).
• Triple Extraction (TE): Identifying entities and the semantic relations between them (e.g., CMeIE-v2).
• Clinical Event Extraction (CEE): Detecting event triggers, types, and arguments (e.g., CHIP-CDEE).
• ICD Encoding: Mapping diagnosis terms to standardized ICD-10 codes (e.g., CHIP-CDN).

Evaluation Framework:
To move beyond simple accuracy, the study evaluated ChatGPT across five distinct dimensions:

1. Performance: Measured using instance-level strict F1-scores.
2. Explainability: Assessing the quality of reasoning provided by the model at both sample and instance levels (R-score).
3. Faithfulness: Checking "Instruction Following" (adherence to constraints) and "Faithful Reasoning" (alignment between explanations and the original text).
4. Confidence: Investigating whether the model's self-reported confidence aligns with its actual correctness (CC-score vs. IC-score).
5. Uncertainty: Measuring the variance in results across multiple runs (5 queries per sample) due to the top-p sampling decoding algorithm.

Experimental Setup:
The researchers used a closed MedIE setting via carefully designed task prompts (including task descriptions, label sets, output formats, and few-shot demonstrations). Results were compared against fine-tuned baseline models (BERT, UIE) and State-of-the-Art (SOTA) methods.

3. Key Contributions

• Comprehensive Benchmarking: One of the first systematic studies to evaluate LLMs on a wide array of fine-grained, specialized medical extraction tasks.
• Multi-dimensional Analysis: Moving beyond F1-scores to analyze the "black box" aspects of LLMs, such as explainability, faithfulness, and calibration.
• Expert-Validated Evaluation: Utilizing domain experts for manual annotation to ensure the reliability of qualitative metrics (explainability and faithfulness).
• Open Science: The authors made their code publicly available to facilitate future MedNLP research.

4. Results

• Performance Gap: ChatGPT’s performance significantly lags behind fine-tuned baseline models and SOTA methods across all tasks. It performs better on simpler tasks (like ICD coding and flat NER) but struggles with complex tasks like Triple Extraction (CMeIE-v2).
• High Explainability but Over-confidence: ChatGPT provides high-quality, human-approved explanations (R-scores > 75%). However, it exhibits an over-confidence phenomenon: it provides confident-sounding explanations even when its predictions are incorrect and fails to reflect on its mistakes.
• Strong Faithfulness: The model demonstrates high levels of instruction following and faithful reasoning (mostly > 80%), suggesting that its errors are not due to ignoring instructions but rather due to a lack of domain-specific precision.
• High Uncertainty: Due to the stochastic nature of LLM decoding, the model produces inconsistent results across different runs, which poses a risk for clinical applications requiring high reliability.

5. Significance

This paper serves as a critical reality check for the application of LLMs in the medical domain. It highlights that while ChatGPT is a "reasonable" conversationalist and a capable "reasoner" (as evidenced by its explanations), it is currently not a reliable substitute for fine-tuned, task-specific models in medical information extraction. The findings regarding over-confidence and uncertainty are particularly significant for medical safety, suggesting that developers must implement rigorous calibration and verification layers before deploying such models in clinical decision-support systems.