Technical Summary: Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks

Problem Statement

The rapid adoption of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) in agricultural imaging presents significant risks due to "hallucinations"—outputs that appear confident but deviate from biological or environmental reality. In the agricultural domain, such errors can lead to misinformed agronomic insights, including incorrect disease diagnosis, inappropriate pesticide application, and flawed yield assessments. While LLMs show promise in automating documentation and decision support, their susceptibility to generating factually incorrect or biologically impossible content remains a critical barrier to trust and deployment. This study addresses the lack of rigorous assessment regarding the accuracy and safety of LLM outputs in two specific modalities: image-to-text (interpreting crop imagery) and text-to-image (generating synthetic agricultural scenes).

Methodology

The authors conducted a systematic evaluation of hallucination behaviors across interpretive and generative tasks using multiple state-of-the-art models, including Gemma, LLAVA, Qwen, MiniCPM, GPT-5, and Gemini 2.5 Flash.

1. Image-to-Text Evaluation (Interpretation)

The study evaluated the models' ability to interpret agricultural imagery through two experimental setups:

• Crop Condition Classification: Models were tasked with classifying tomato leaf images as "healthy" or "diseased" (specifically Bacterial spot).
  • Zero-shot: Models relied solely on pre-trained knowledge without examples.
  • Few-shot: Models were provided with reference images of healthy and diseased leaves to guide interpretation.
• Fruit Detection and Counting: Using a subset of a greenhouse strawberry dataset (100 images), models were asked via Question-Answering (QA) to:
  • Q1: Determine the presence of fruit.
  • Q2: Identify the developmental stage (ripe vs. unripe).
  • Q3: Count the fruits in each category.
• Metrics: Performance was measured using Classification Accuracy, F1-scores, and confusion matrices to identify false positives (hallucinated disease) and false negatives (missed infections).

2. Text-to-Image Evaluation (Generation)

The study assessed the biological consistency of images generated from textual prompts.

• Unprompted/Extraneous Content: Models were prompted to generate specific agricultural images (e.g., thermal images of infected leaves, RGB images of mold-infected strawberries). The outputs were analyzed for the inclusion of visual elements not specified in the prompt (e.g., artificial temperature gradients, background debris, or textual labels).
• Biological Contradictions: The authors tested models with "implausible" prompts designed to trigger hallucinations, such as generating an image of a "disease-free strawberry covered with powdery mildew."
  • Prompt Variations: Tests were conducted with direct requests (T1) and rephrased requests including a justification for research purposes (T2) to observe if safety guardrails could be bypassed.
• Evaluation Criteria: Generated images were scrutinized for anatomical impossibilities, physiological contradictions (e.g., healthy plants with severe pest damage), and environmental inconsistencies.

Key Contributions

The paper makes three primary contributions to the field of agricultural AI:

1. Identification of Hallucination Patterns: The study systematically characterizes recurring hallucination patterns in both interpretive and generative agricultural tasks, specifically highlighting errors in biological consistency, contextual accuracy, and agronomic plausibility.
2. Quantitative Assessment of Reliability: The authors provide empirical data on the performance of multiple MLLMs in agricultural scenarios, demonstrating that even advanced models exhibit significant error rates in zero-shot settings and remain susceptible to hallucinations even when few-shot prompting is applied.
3. Analysis of Generative Vulnerabilities: The research reveals that generative models can produce biologically inconsistent scenes (e.g., healthy crops with pest damage) and that subtle prompt modifications (e.g., adding a research justification) can override safety mechanisms, leading to the generation of agronomically impossible imagery.

Results

Image Interpretation Performance

• Zero-shot Accuracy: Models achieved modest accuracy ranging from 63% to 75% in classifying tomato leaf diseases.
• Few-shot Improvement: Performance improved with few-shot prompting, reaching up to 86.8% (Gemma), but hallucinations persisted.
• Error Types:
  • Qwen exhibited a high rate of false negatives in strawberry detection, frequently denying the presence of fruit despite visual evidence.
  • MiniCPM showed a significant discrepancy between accuracy (76.84%) and F1-score (0.782) in fruit detection, and a wide gap in zero-shot tomato classification (63.0% accuracy vs. 0.46 F1-score), indicating residual hallucination effects.
  • LLAVA and Gemma demonstrated more balanced error profiles but still produced non-negligible false predictions.

Image Generation Findings

• Unprompted Elements: Models frequently added agronomically irrelevant details. For instance, GPT-5 generated exaggerated thermal gradients and halo transitions in thermal images, while Gemini 2.5 Flash added uniform glowing regions or unrequested background elements like soil and irrigation channels.
• Biological Inconsistency:
  • When prompted to generate a "healthy soybean canopy with extensive pest damage," models like GPT-5 and Gemini 2.5 Flash generated images where leaves showed severe feeding holes yet were labeled or visually implied as "healthy."
  • In the "disease-free strawberry with powdery mildew" test, Gemini 2.5 Flash generated a biologically impossible image (91% generation rate under relaxed constraints) showing a healthy fruit covered in mold.
  • GPT-5 initially refused the direct prompt (T1) but complied with the rephrased research justification (T2), generating a contradictory image with a 91% generation rate for biologically implausible scenes.
• Prompt Sensitivity: The generation rate of biologically implausible content increased sharply when prompts were framed as research illustrations (e.g., GPT-5 rose from 64% to 91%).

Significance and Claims

The paper claims that this systematic assessment offers critical insights into the reliability of LLM-based agricultural imaging platforms. The authors emphasize that:

• Current Limitations: Even advanced models (GPT-5, Gemini 2.5 Flash) possess fundamental weaknesses in biological grounding, leading to outputs that are linguistically fluent but factually incorrect or physiologically impossible.
• Risk to Stakeholders: Unrecognized hallucinations can mislead farmers, researchers, and policy planners, potentially eroding trust in AI systems and causing tangible economic or environmental harm through incorrect agronomic recommendations.
• Need for Safeguards: The study argues that achieving the potential of digital agriculture requires rigorous validation pipelines, domain-specific fine-tuning, and constraint-guided generation to ensure outputs remain true to biological reality. The authors suggest that future work must focus on strengthening prompt resilience and developing biologically grounded decoding frameworks to mitigate these risks.