QMSR: Query-Conditioned Mask-wise Expert Routing for Robust Open-Vocabulary Underwater Object Retrieval
The paper proposes QMSR, a query-conditioned mask-wise expert routing framework that adaptively selects and fuses pretrained underwater image enhancement experts with raw representations for each query-candidate pair, thereby overcoming the limitations of fixed enhancement strategies and significantly improving open-vocabulary object retrieval performance in complex underwater environments.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Deep beneath the surface, the ocean is a place where light behaves differently than it does on land. Water absorbs colors, scatters light, and turns the world into a murky, low-contrast haze. For robots exploring these depths, this visual distortion is a major obstacle. To perform useful tasks, like picking up a specific tool or identifying a piece of debris, a robot needs to "see" clearly. In recent years, scientists have developed powerful computer systems that allow machines to understand images through language. A human can simply type a request like "find the red wrench," and the computer can scan an image to locate it. However, this technology struggles underwater. The hazy, color-shifted images produced by the ocean often confuse the computer, making it difficult to match the visual scene with the words describing it.
For a long time, the standard solution to this problem was to try to fix the image first. Researchers have created many different software tools designed to clean up underwater photos, sharpening edges and restoring natural colors before the robot tries to identify objects. The assumption was simple: a clearer picture should always lead to a better answer. But this approach has a hidden flaw. The ocean is not uniform; some parts of an image might be blurry while others are clear, and different objects might rely on different visual clues. A software tool that makes the whole image look brighter might accidentally wash out the specific color or texture a robot needs to find a target. In fact, the researchers behind this new study found that applying these standard fixes to every single image often made the robot's search worse, not better. Sometimes, the original, unedited image actually contained the best clues for finding the right object.
To solve this, a team of researchers from Nanyang Technological University and The Chinese University of Hong Kong developed a new system called QMSR. Instead of forcing every image through a single cleaning process, their system acts like a smart decision-maker that evaluates each potential object individually. When the robot receives a command, such as "find the plastic cup," the system first breaks the image down into many small candidate pieces, or masks, that could contain the object. For each of these pieces, the system asks a crucial question: "Does this specific piece of the image need to be cleaned up to match the words I am looking for?"
The system has access to a library of nine different image-cleaning experts, each trained to fix underwater photos in a slightly different way. Some might be better at restoring blue tones, while others excel at sharpening edges. The new framework does not pick one expert for the whole image. Instead, it looks at the specific object the robot is hunting for and the specific piece of the image it is examining. It then selects the single best cleaning expert for that specific pair. If the system decides that a particular piece of the image is already clear enough, or that cleaning it would actually hide the clues it needs, it chooses to leave that piece exactly as it is. This is a critical distinction: the system learns to know when not to enhance an image, preserving the raw information that might be vital for the search.
The researchers tested this approach using a large collection of underwater images and text queries. They compared their new method against the old way of using a single cleaning tool for everything, and against using no cleaning at all. The results were striking. The new system improved the robot's ability to find the correct objects by a significant margin, outperforming the best fixed cleaning strategy by nearly twenty-six percent. It also proved to be very flexible; when the researchers tested it with words and objects it had never seen before during its training, it still performed much better than the traditional methods. This suggests the system learned a general rule about how to match visual clues with language, rather than just memorizing specific fixes.
Perhaps the most surprising discovery was that the system did not need to be told which cleaning method was best during its training. Instead, it learned by observing the final results of its choices. The researchers used a special training technique where a "teacher" program, which had access to all the answers, guided the system on which expert to pick for each object. Over time, the system learned to make these decisions on its own, using only the raw image and the text command. It turned out that for almost every situation, picking just one expert and deciding how strongly to apply their fix was enough to capture the benefits of having all nine experts available. The system learned that a one-size-fits-all approach was the problem, and that the key to seeing clearly underwater was knowing exactly which tool to use, and when to use it, for every single object in sight.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.