MedPT: A Massive Medical Question Answering Dataset for Brazilian-Portuguese Speakers

1. Problem Statement

The development of Large Language Models (LLMs) for healthcare is heavily skewed toward high-resource languages, particularly English. This creates a critical barrier for Portuguese speakers due to two main factors:

• Linguistic and Cultural Gaps: Simple translation of existing English datasets fails to capture unique clinical nuances, region-specific colloquialisms, and the epidemiology of endemic diseases prevalent in Brazil (e.g., Dengue, Chagas disease).
• Data Scarcity: Existing Portuguese medical datasets are either limited to formal clinical notes (hospital records) or are small-scale, proprietary, or focused on narrow domains (e.g., women's health). There is a lack of large-scale, open-source corpora containing authentic, open-ended patient-doctor interactions in Brazilian Portuguese.

2. Methodology

The authors developed MedPT, a massive dataset derived from real-world interactions, through a rigorous multi-stage pipeline:

A. Data Acquisition

• Source: Data was scraped from Doctoralia, a major Brazilian platform where patients ask questions and verified doctors provide answers.
• Volume: Approximately 2 million raw samples were initially collected.
• Privacy: All data is anonymous, ensuring user confidentiality.

B. Data Cleaning and Curation

A hybrid quantitative-qualitative protocol was applied to reduce noise and enhance quality, resulting in 384,095 unique question-answer pairs:

1. Deduplication: Removed redundant entries caused by the platform's design (where one Q&A pair was linked to multiple specialties).
2. Textual Normalization: Stripped HTML tags, URLs, and special characters.
3. Column-Specific Refinement:
   • Questions: Contextually enriched 7,178 short, ambiguous queries (e.g., "What is the treatment?") by prepending the associated medical condition to make them self-contained.
   • Answers: Filtered out uninformative answers (<10 tokens, e.g., "Consult a specialist") and overly generic articles (>1,000 tokens), retaining a valid range of 10–1,000 tokens.
   • Conditions & Specialties: Removed malformed data and non-specialty artifacts (e.g., city names) while preserving valid multi-specialty entries.

C. Semantic Annotation

To capture user intent, the dataset was enriched using LLM-driven annotation (specifically gpt-oss-120b). Each question was classified into one of seven semantic types:

1. Anatomy and Physiology
2. Choosing Healthcare Professionals
3. Diagnosis
4. Epidemiology
5. Healthy Lifestyle
6. Treatment
7. Other

3. Key Contributions

• First Large-Scale Native Corpus: MedPT is the first public, large-scale dataset of authentic patient-doctor interactions specifically for the Brazilian Portuguese medical domain.
• Scale and Diversity:
  • 384,095 high-quality Q&A pairs.
  • ~57 million tokens total.
  • Covers 3,200+ distinct medical conditions across 104 medical specialties.
  • Includes metadata on disease, specialty, and question type.
• Rigorous Curation: The paper details a novel hybrid curation process that combines statistical outlier detection with manual inspection and LLM-based contextual enrichment, setting a standard for dataset construction in low-resource languages.
• Public Release: The dataset is publicly available on Hugging Face to foster equitable AI development.

4. Experimental Results

To validate the dataset's utility, the authors benchmarked it on a Medical Specialty Classification task (predicting the correct medical specialty based on a patient's question).

• Model: Qwen3 (1.7B parameters).
• Tasks: Classification across Top 5, Top 10, and Top 20 most frequent specialties.
• Performance:
  • Fine-Tuning: The model fine-tuned on MedPT achieved an outstanding 94% F1-score on the challenging 20-class setup (compared to 60% for the zero-shot baseline).
  • Few-Shot Learning: In-context learning (3-shot) significantly boosted performance (from 0.60 to 0.73 F1 in the 20-class task), demonstrating the dataset's high signal-to-noise ratio.
• Error Analysis: Misclassifications were not random; they occurred between semantically and clinically related conditions (e.g., Anxiety vs. Depression, or Hernia types), indicating the model learned meaningful clinical relationships and that the dataset reflects genuine clinical ambiguities.

5. Significance

• Bridging the Resource Gap: MedPT directly addresses the linguistic bias in medical AI, enabling the development of models that understand Brazilian cultural contexts, endemic diseases, and local healthcare systems.
• Clinical Reliability: By capturing real-world patient language (including colloquialisms) and verified doctor responses, the dataset supports the creation of more accurate, safe, and trustworthy medical AI tools.
• Foundation for Future Research: The dataset enables diverse downstream tasks, including automated inquiry routing, patient intent classification, and the training of specialized medical LLMs for the Portuguese-speaking world (Brazil and Portugal).
• Ethical Impact: The authors emphasize that these tools are designed to assist professionals, not replace them, highlighting the need for human oversight in medical decision-making.