VisText-VLM2Plastic: A Bilingual Benchmark for Visual Grounding and Environmental Reasoning in Vision-Language Models
This paper introduces VisText-VLM2Plastic, a bilingual English–Bangla benchmark for plastic-waste visual grounding and environmental reasoning, revealing that while state-of-the-art vision-language models achieve high recognition accuracy, they exhibit significant performance gaps between languages and varying reliance on visual evidence across different task types.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where computers can look at a photograph and understand not just what objects are present, but also the story those objects tell about our environment. This is the promise of vision-language models, a type of artificial intelligence that combines the ability to see images with the ability to read and write language. For years, researchers have tested these systems with questions like "What color is the car?" or "How many people are in the room?" However, a critical gap has emerged: we do not know if these models are truly looking at the picture to answer, or if they are simply guessing based on the words in the question. This is especially important when the questions involve complex topics like pollution, where a computer might know general facts about plastic waste without ever needing to see the specific trash in front of it. Furthermore, most of these tests have been conducted in English, leaving a huge gap in understanding how these systems perform in other major languages, such as Bangla, which is spoken by hundreds of millions of people but remains underrepresented in digital research.
To address these issues, a researcher at United International University created a new, controlled test designed to see if artificial intelligence can truly "ground" its answers in visual evidence, specifically regarding plastic waste. The study, titled VisText-VLM2Plastic, focused on a bilingual benchmark, meaning it tested the same set of images and questions in both English and Bangla. The researchers gathered 136 distinct photographs of plastic litter found in various locations, which were then used to create 402 image records. From these images, they generated over 2,400 question-and-answer pairs. These questions fell into three categories: simple recognition (identifying the type of plastic), visual description (describing what the object looks like), and environmental reasoning (explaining why the waste is harmful to the environment). The goal was to see if the models could answer correctly in both languages and, crucially, whether they would fail if the image was removed, proving they were relying on the picture rather than just memorized facts.
The researchers tested four different advanced artificial intelligence models on this new benchmark, asking them to answer the questions without any prior training on the specific dataset. The results revealed a clear hierarchy in performance. One model, Gemini 2.5 Flash, achieved the highest accuracy, correctly identifying the type of plastic waste in 93.5 percent of cases. However, a significant pattern emerged when comparing the two languages: across all models, performance in English was consistently higher than in Bangla. The gap was most noticeable in the environmental reasoning tasks, where the English scores were about four points higher on a normalized scale than the Bangla scores. This suggests that while these systems are becoming quite capable, they still struggle to apply the same level of understanding when switching to a language like Bangla, which has fewer digital resources available for training.
Perhaps the most revealing finding came from a specific experiment where the researchers removed the images entirely, leaving the models with only the text of the questions. When asked to identify the plastic or describe its appearance, the models' performance collapsed dramatically without the visual input, dropping by more than 50 points. This confirmed that for these tasks, the models were indeed looking at the image. However, the story changed for the environmental reasoning questions. When the image was removed, the models' ability to answer questions about the dangers of plastic pollution dropped by only about 11 points. This small gap indicates that for these specific questions, the models were not strictly dependent on the image. Instead, they were able to provide plausible answers based on their general knowledge of plastic pollution, such as the fact that it clogs drains or harms wildlife, even when they could not see the specific bottle or wrapper in the photograph.
The study concludes that simply measuring how often a model gets the right answer is not enough to judge its intelligence. A model can give a correct answer about environmental damage without actually understanding the visual evidence in front of it. The researchers found that for a model to be truly reliable in this field, it must be tested not just on its final score, but on its consistency across different languages and its sensitivity to visual input. The work highlights that while artificial intelligence is making strides in understanding our world, there is still a long way to go before these systems can be trusted to analyze environmental issues in all languages with the same depth and reliance on real-world evidence. The benchmark serves as a diagnostic tool, showing that future systems need to be designed to force a stronger connection between what they see and what they say, ensuring they do not just recite facts but truly observe the world around them.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.