Vision Meets Language: A RAG-Augmented YOLOv8 Framework for Coffee Disease Diagnosis and Farmer Assistance

Imagine the world of farming as a giant, living puzzle where every leaf, stem, and drop of water plays a crucial role. For centuries, farmers have been the detectives of this puzzle, trying to figure out why a plant is sick and how to cure it without hurting the environment. Today, we are in the middle of a scientific revolution where computers are learning to help solve these puzzles. This field, known as "precision agriculture," uses high-tech tools to make farming smarter and greener. Two of the most powerful tools in this new era are "computer vision" and "artificial intelligence." Think of computer vision as a super-powered pair of eyes that can look at a picture and instantly spot a tiny spot on a leaf that a human might miss. Then there are "Large Language Models" (LLMs), which are like incredibly well-read librarians that can talk to you, answer questions, and write stories, but they sometimes make things up if they aren't careful. The big question scientists are asking is: Can we combine these super-eyes with the super-librarian to create a farming assistant that doesn't just spot a disease, but also explains exactly what's wrong and how to fix it, all while keeping the environment safe?

This paper introduces a clever new system designed to help coffee farmers by acting as a digital doctor for their plants. The researchers built a hybrid framework that teams up three different AI technologies to tackle the problem of coffee leaf diseases. First, they use a computer vision model called YOLOv8 (which stands for "You Only Look Once," version 8). You can think of YOLOv8 as a lightning-fast security guard that scans a photo of a coffee leaf and instantly points out, "Hey, there's a disease here!" It draws a box around the sick part and names the trouble. However, a security guard who just shouts "Thief!" without explaining who they are or how to catch them isn't very helpful. That's where the second and third parts come in.

To fix the "just shouting" problem, the team added a Retrieval-Augmented Generation (RAG) system. Imagine the YOLOv8 guard finding a sick leaf and then running to a massive, organized library of agricultural books. Instead of guessing the cure, the guard asks the librarian (the RAG system) to find the specific pages about that exact disease. This step is crucial because it stops the AI from making up fake cures, a problem known as "hallucination" where AI sometimes invents facts. Finally, a Large Language Model (LLM) acts as the friendly translator. It takes the disease name from the guard and the specific facts from the library, then writes a clear, easy-to-understand message for the farmer. It says, "Your coffee leaf has Rust. Here is exactly what caused it, and here are three safe ways to treat it using methods that won't harm the environment."

The researchers tested this system using a dataset of coffee leaf images called BRACOL, which contains thousands of pictures of healthy and sick leaves. They trained their "security guard" (YOLOv8) on these images and found that it could successfully spot four different types of coffee diseases. In their tests, the system was particularly good at spotting "Miner" and "Phoma" diseases, achieving a high level of accuracy. However, it found "Rust" a bit trickier to identify, likely because there were fewer pictures of it in the training data and it looks a bit like healthy leaf textures. The system didn't just stop at detection; when a farmer uploaded a picture, the AI successfully retrieved real information about the disease and generated a helpful, step-by-step remedy plan. The team also built a chat feature, so if a farmer asked, "What if I wait a week to treat this?" the system could answer based on the retrieved facts, not just a guess.

The main finding of this work is that combining these three tools—vision, retrieval, and language generation—creates a much more reliable and useful tool than using any of them alone. The paper suggests that this approach helps reduce the need for guesswork and excessive pesticide use, offering a way to save crops while protecting nature. While the current version works well, the author notes that the system could be even better if they used newer versions of the vision model or added more expert knowledge to the library. Ultimately, this paper shows that by letting AI "look" at the problem and then "read" the solution from a trusted source, we can build smarter tools that help farmers take care of their crops and the planet.