🧬 biology

AI-generated seed images do not outperform matched reuse of scarce real defect images in seed quality assessment

This study demonstrates that for automated seed defect inspection, simply increasing the frequency of scarce real defect images through balanced resampling consistently outperforms the use of AI-generated synthetic images, suggesting that observed performance gains are often attributable to improved class exposure rather than the added content of synthetic data.

Original authors: Bhanu Prakash Vangala, Navya Vangala

Published 2026-10-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Bhanu Prakash Vangala, Navya Vangala

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the world of agriculture, the journey of a seed from the field to the store shelf is a race against imperfection. Farmers and processors rely on high-speed cameras and automated systems to sort seeds, separating the healthy, plump ones from those that are broken, immature, or spotted with disease. These visual defects matter deeply; a broken seed might not sprout, and a diseased one could spread infection through a storage bin. To teach a computer to spot these flaws, engineers need thousands of examples. They feed the software images of perfect seeds and thousands of images of damaged ones. But nature does not cooperate with this demand for volume. While healthy seeds are easy to find in vast numbers, the specific kinds of damage that matter most—like a rare crack or a specific type of rot—are often scarce. A researcher might have ten thousand pictures of good soybeans but only a handful of images showing a particular, rare defect. This lack of data makes it hard to train a computer to recognize the problem it is supposed to solve.

To fix this shortage, scientists have turned to artificial intelligence to create fake images. Using powerful computer models, they can generate new pictures of damaged seeds that look real, hoping to fill the gaps in their training data. The idea is simple: if you cannot find enough real broken seeds, make more of them. This approach has become popular, with many studies claiming that adding these synthetic images helps computers learn better. However, a new study from researchers at the University of Missouri and Amar Biotech India suggests that the story is more complicated. They asked a sharper question than anyone had before: did the computer get better because it learned from the new, fake images, or did it simply get better because it saw the few real images it already had more often?

The researchers set up a careful experiment using soybean seeds, focusing on four types of rare defects: broken seeds, immature ones, skin-damaged seeds, and spotted seeds. They started by creating a difficult scenario where the computer had access to very few examples of each defect, simulating the real-world problem of data scarcity. They tested four different levels of scarcity, giving the computer as few as twenty-five real images of each defect type, and as many as two hundred. In the first part of the test, they added hundreds of AI-generated images to the training set. As expected, the computer's performance improved significantly. When the computer had only twenty-five real images to start with, adding the fake ones boosted its ability to identify defects by more than fifty percentage points. This confirmed the usual belief that synthetic data helps.

But the researchers did not stop there. They wanted to know if the improvement came from the content of the fake images or simply from the quantity of data. To find out, they ran a second, parallel test. In this version, they did not add any new images. Instead, they took the same small number of real defect images they had and showed them to the computer many more times, effectively repeating the same few examples until the total number of training pictures matched the size of the group that received the fake images. This was a fair comparison: both groups saw the same number of pictures, but one group saw new, generated pictures while the other saw the same real pictures over and over again.

The result was surprising. The group that simply saw the real images repeated did just as well, or even better, than the group that saw the AI-generated images. In fact, when the researchers measured the specific benefit of the fake images by subtracting the performance of the repeated-real-images group from the fake-image group, the result was negative. The AI-generated images did not add any extra value. In the most extreme case, where the computer started with only twenty-five real images, the fake images actually performed eight percentage points worse than simply reusing the real ones. The improvement seen in the first test was not because the AI had invented new, useful examples of damage; it was because the computer had simply been exposed to the rare defect categories more frequently.

To ensure this was not a fluke, the researchers tried several other tricks. They tested different types of computer vision systems, ranging from simple networks to more complex, modern architectures. In every case, the pattern held: simply showing the computer the real defects more often worked better than feeding it synthetic data. They even tried a "stress test" where they added fake images with completely wrong labels—telling the computer that a broken seed was actually a spotted one. Even with this confusing, incorrect information, the computer's performance still improved when the data was scarce. This proved that the computer was so desperate for data that it improved just by seeing more rows of data, even if some of that data was misleading. The real gain came from balancing the training set, not from the quality of the new images.

The study also looked at whether filtering the fake images could save the day. Perhaps the AI was generating too many poor-quality pictures, and if they were cleaned up, they would help. The researchers tried various methods to select only the best-looking fake seeds, but even after filtering, the synthetic images failed to outperform the simple strategy of reusing the real ones. They repeated the entire experiment with a different crop, maize, where the labels described different varieties rather than defects. The result was the same. The advantage of synthetic data disappeared when the researchers controlled for how often the computer saw the rare classes.

The takeaway is not that artificial intelligence cannot help in agriculture, but that its role needs to be understood more clearly. The study shows that when data is scarce, the biggest problem is often not a lack of variety in the images, but a lack of exposure to the rare categories. A computer learns to recognize a broken seed not because it has seen a million different versions of a crack, but because it has seen the few real cracks enough times to understand what they look like. When researchers claim that synthetic images have added new biological information or recovered missing variations, this study suggests they may be mistaken. The improvement comes from the act of rebalancing the data, not from the generation of new content. For scientists and engineers building these systems, the lesson is to be careful about what they credit to the AI. Before celebrating the power of synthetic data, they should first check if simply showing the computer the real, scarce examples more often would have done the job just as well.

Technical Summary: AI-Generated Seed Images vs. Matched Real-Data Reuse in Quality Assessment

Problem Statement
In automated seed quality assessment, particularly for crops like soybeans, training data for rare defect classes (e.g., breakage, incomplete filling, spotting) is often scarce. This scarcity leads to class imbalance, where classifiers underperform on minority defect classes. While synthetic image generation (e.g., using Generative AI) is frequently proposed to augment these datasets, standard evaluations typically compare an augmented model against a weak, un-augmented baseline. This approach conflates two distinct factors: the introduction of new image content and the increased frequency of exposure to rare classes during training. Consequently, it remains unclear whether performance gains are due to the generative model recovering missing biological variation or simply resulting from better class balancing and increased training iterations on rare examples.

Methodology
The authors propose a matched-control experimental design to isolate the specific contribution of generated image content from the general benefits of class rebalancing.

Experimental Design: The study utilizes a soybean dataset with five classes (Intact, Broken, Immature, Skin-damaged, Spotted). Scarcity is simulated by retaining only $n \in \{25, 50, 100, 200\}$ real images per defect class for training, while the majority class remains fully represented.
Control Arms:
1. Synthetic Augmentation ( $G_k$ ): A class-conditional Denoising Diffusion Probabilistic Model (DDPM) generates new defect images.
2. Matched Real Reuse ( $R_k$ ): An equal number of real defect images are added to the training set via sampling with replacement from the existing scarce pool.
3. Other Controls: Balanced resampling ( $B_n$ ), inverse-frequency loss weighting ( $W_n$ ), and a "wrong-label" stress test ( $R^\pi_k$ ) where labels of duplicated images are permuted to test baseline sensitivity to row count alone.
Metrics: Performance is measured using Macro-F1 on a fixed, held-out set of real seeds. The core metric is the decomposition of the total augmentation gain ( $\Delta_{aug}$ $Δ_{a ug}$ ) into:
- $\Delta_{reuse}$ : The gain from matched real-data reuse.
- $\Delta_{info}$ : The residual gain attributed specifically to generated content ( $\Delta_{aug} - \Delta_{reuse}$ ).
Robustness Checks: The study evaluates five different classifier architectures (Swin-T, ViT-B/16, ConvNeXt-T, EfficientNetV2-S, ResNet-50), applies filtering to generated images based on confidence and feature-space typicality, and replicates the logic on a maize dataset with different label semantics (varietal identity vs. defects).

Key Results

No Advantage from Generated Content: Across all scarcity levels, the matched reuse of real defect images ( $\Delta_{reuse}$ ) consistently outperformed or equaled the synthetic augmentation ( $\Delta_{aug}$ ). The residual gain attributed to generated content ( $\Delta_{info}$ ) was never positive, ranging from -8.05 to -0.61 percentage points depending on the scarcity level.
Baseline Sensitivity: In the most severe scarcity conditions ( $n=25$ ), even the "wrong-label" control (adding images with permuted, incorrect labels) improved the baseline performance by ~34 percentage points. This indicates that the observed gains in standard augmentation studies are often driven by the sheer increase in training rows and class exposure rather than the quality or novelty of the data.
Classifier Dependency: The magnitude of augmentation gains was inversely correlated with the baseline performance of the classifier. Weaker models showed larger apparent gains, suggesting that the "benefit" of augmentation is partly a function of the model's initial capacity to learn from scarce data, not just the data itself.
Filtering Limitations: Filtering generated images to remove low-quality candidates did not recover a consistent advantage. Furthermore, the classifiers used to score and filter synthetic images were themselves unreliable at the lowest data levels ( $n=25$ ), creating a circular dependency.
Cross-Species Validation: The findings held true for a maize dataset, confirming that the lack of advantage for synthetic data over matched real reuse is not specific to the soybean defect taxonomy.

Significance and Claims
The paper argues that the scientific community must distinguish between the general utility of augmentation (which improves weak baselines) and the specific claim that generative AI recovers missing biological variation.

Reframing the Narrative: The authors assert that if matched reuse of real defects or simple resampling strategies yield equal or superior performance, the observed improvement should not be interpreted as evidence that the generator has successfully modeled unseen defect variations.
Methodological Standard: The study proposes a stricter evaluation framework for future synthetic seed image research. Before claiming that new information has been added, researchers must demonstrate that the improvement persists after controlling for class exposure and rebalancing.
Modest Conclusion: The paper does not claim that synthetic images are useless for all purposes (e.g., simulating acquisition conditions or instrument development). However, it concludes that for the specific task of training classifiers to recognize rare defects, the "generated content" does not provide a measurable advantage over simply using the available real data more effectively. The central claim is that the apparent benefits of synthetic augmentation in this domain are largely attributable to the mechanics of class rebalancing rather than the generative model's ability to synthesize novel, biologically valid seed variations.

Technical Summary: AI-Generated Seed Images vs. Matched Real-Data Reuse in Quality Assessment

More like this