Technical Summary: LeafMD – A Multiregional Image–Text Dataset and Benchmark for Vision-Language Modeling of Plant Diseases

Problem Statement

Plant diseases pose a critical threat to global food security, causing estimated annual losses of USD 220 billion. While artificial intelligence and deep learning offer potential for scalable disease detection, current approaches are hindered by the limitations of existing datasets. Most public plant disease resources suffer from three primary deficiencies:

1. Limited Geographic and Environmental Diversity: Existing datasets are often fragmented, dominated by controlled-environment imagery (laboratory settings), and restricted to a narrow range of crop species or geographic regions. This limits the transferability and scalability of models trained on them to real-world agricultural conditions.
2. Lack of Semantic Depth: Annotations are typically limited to categorical disease labels for image classification. They lack biologically grounded semantic information, such as detailed symptom descriptions, disease progression stages, or observation contexts, which are crucial for fine-grained pathological reasoning.
3. Absence of Standardized Multimodal Benchmarks: There is a lack of interoperable resources that integrate image-text pairs to support advanced tasks like visual question answering (VQA), symptom interpretation, and disease staging under realistic field conditions.

Methodology

The authors present LeafMD, an integrated resource comprising LeafNet 2.0 (a large-scale image-text dataset) and LeafBench 2.0 (a visual question answering benchmark). The construction followed a three-stage workflow:

1. Image Dataset Collection (LeafNet 2.0)

• Expansion: The dataset was built by expanding the original LeafNet collection, incorporating 89,398 additional images from South and Southeast Asia to complement existing data from Africa and the US.
• Scale and Diversity: The final dataset contains 255,855 image-text pairs covering 37 crop species, 197 crop-disease classes, and 9 geographic regions (spanning tropical, subtropical, and temperate systems).
• Curation: Images were sourced from public repositories and direct institutional communications. A rigorous quality control process excluded images with unclear symptoms, negligible lesion areas, non-leaf focus (e.g., fruits/stems), or multiple co-infections (to avoid annotation uncertainty). All retained images were resized to 224 × 224 pixels.
• Realism: The extended collection significantly increased the proportion of images captured under natural field conditions with heterogeneous backgrounds and variable illumination, reducing the dominance of controlled-environment imagery.

2. Caption Construction

• Human-in-the-Loop Annotation: Trained annotators created class-level reference descriptions based on peer-reviewed literature (e.g., Springer Nature, Elsevier) and agricultural extension reports. These descriptions focused on diagnostically relevant traits (lesion morphology, color, distribution, deformation) for early and late disease stages.
• LLM-Assisted Generation: To ensure consistency and scale, OpenAI GPT-4 was used to generate 255,855 image-specific captions. The model was guided by the human-authored reference captions to integrate visual context with biological descriptions.
• Quality Control: A three-model evaluation pipeline (using Gemini Pro 2.5, GPT-4, and Claude 3.5) assessed the clarity and relevance of human-authored references. Subsequently, generated captions were validated by domain experts. Samples with inconsistencies (approx. 7%) were revised or excluded.

3. Benchmarking (LeafBench 2.0)

• Task Design: LeafBench 2.0 is a multiple-choice VQA benchmark derived from LeafNet 2.0, covering nine fine-grained tasks: Disease Identification (DI), Pathogen Classification (PC), Crop Species Identification (CSI), Symptom Identification (SI), Healthy–Diseased Classification (HDC), Scientific Name Classification (SNC), Lesion Identification (LI), Leaf Symptom Detection (LSD), and Disease Severity Classification (DSC).
• Evaluation: The benchmark was tested on 16 vision-language models, including CLIP-based architectures, generative VLMs, and proprietary models (GPT-4o, Gemini 2.5 Pro).

Key Results

Dataset Validation

• Biological Accuracy: Expert validation on 6,361 image-caption pairs showed 100% agreement on crop species and disease labels.
• Stage Annotation: Disease stage annotations achieved 99% consistency where visual evidence was sufficient.
• Caption Quality: While 7% of samples required revision due to generic descriptions or partial symptom matches, the pipeline successfully preserved core biological identity and disease class consistency across the dataset.

Benchmark Performance

• Performance Gaps: A significant performance gap exists between coarse disease recognition and fine-grained pathological reasoning.
  • High Performance: Models excelled at Healthy–Diseased Classification (HDC) (up to 95.70% with Gemini 2.5 Pro) and Crop Species Identification (CSI) (up to 76.80%).
  • Low Performance: Tasks requiring subtle discrimination, such as Pathogen Classification (PC), Scientific Name Classification (SNC), and Lesion Identification (LI), yielded much lower accuracy (often below 50% for open-source models).
• Model Comparison:
  • Proprietary Models: Gemini 2.5 Pro (67.78% average) and GPT-4o (64.72% average) demonstrated the most balanced performance across all tasks.
  • Domain-Specific Models: Agriculture-adapted models, particularly AgriCLIP and SCOLD, consistently outperformed several larger general-domain architectures on symptom-oriented tasks (e.g., AgriCLIP achieved 65.10% on Disease Identification).
  • Open-Source Models: Among open-source generative models, Qwen3-VL-4B achieved the highest average accuracy (60.02%).

Significance and Claims

The paper claims that LeafMD addresses the critical need for standardized, interoperable, and representative datasets in plant pathology. Its significance lies in:

1. Multimodal Foundation: By integrating biologically grounded symptom descriptions with image-level annotations, LeafNet 2.0 enables the development of disease-aware agricultural foundation models capable of fine-grained pathological reasoning, moving beyond simple categorical classification.
2. Real-World Applicability: The inclusion of diverse geographic regions, crop species, and natural field conditions (including early and late disease stages) provides a more realistic evaluation setting for AI systems intended for operational agricultural deployment.
3. Benchmarking Fine-Grained Reasoning: LeafBench 2.0 establishes a rigorous framework for evaluating the ability of models to interpret subtle symptom variations, infer causal agents, and assess disease severity, revealing current limitations in fine-grained reasoning even for state-of-the-art models.

The authors position LeafMD not merely as a data repository but as a resource to foster the development of interpretable and transferable AI systems that can operate across the variability and uncertainty of real agricultural environments. They note that while current models show promise, significant challenges remain in distinguishing visually similar diseases and performing complex pathological reasoning.