Modern artificial intelligence systems that can see and speak are becoming remarkably capable, yet they carry a heavy computational burden. These vision-language models work by first converting an image into a long sequence of small digital pieces, often called tokens, which represent different parts of the picture. A standard photograph might be broken down into hundreds of these pieces. The system then processes this entire sequence alongside text, much like a human reading a sentence while looking at a diagram. The problem is that the mathematical operations required to connect these pieces grow exponentially as the number of pieces increases. If an image generates 576 tokens, the computer must perform a massive amount of work to understand how they relate to one another, even though many of those tokens represent unimportant background details like a blank wall or a patch of sky. This inefficiency makes it difficult to run these powerful systems on standard hardware or to use them for real-time tasks like video analysis.

Researchers have long tried to solve this by removing the unnecessary pieces before the system gets to work. Some methods act like a filter at the very beginning, discarding tokens based solely on what the image looks like. Others wait until the system has started processing the text and image together, removing pieces that seem irrelevant to the specific question being asked. However, both approaches have a blind spot. The early filters often throw away important details because they don't know what the user is asking, while the later filters waste energy processing the entire image before they ever get to delete anything. A new study introduces a unified approach that combines the strengths of both methods, creating a more efficient path for these intelligent systems to follow.

The researchers, led by Jianhua Cui and Meizhou Ding, developed a framework called UniPrune, which treats token reduction as a two-step journey rather than a single event. Imagine trying to pack a suitcase for a trip where you don't know the exact weather yet. You wouldn't just throw everything in and hope for the best, nor would you try to decide what to keep after you've already packed the whole bag. Instead, you might first group similar items together and keep the most representative ones, then, once you know your destination, you make a final, more precise selection. This is essentially what UniPrune does. It first performs a broad cleanup of the image data using visual patterns, and then, as the system begins to understand the context of a question, it performs a second, sharper cleanup.

The first step of this process happens right after the image is scanned but before the language model gets involved. The system groups the visual tokens into clusters based on their similarity, much like sorting a pile of mixed photos by their general subject matter. From each group, it keeps the most important token and discards the rest. This removes the obvious redundancies, such as multiple tokens representing the same patch of blue sky. However, the researchers realized that simply cutting the number of tokens in half at this stage could be risky. If the system cuts too aggressively too early, it might lose a token that is crucial for answering a specific question later on. To solve this, they introduced a dynamic budgeting strategy. Instead of using a fixed rule for how many tokens to keep at this first stage, the system measures how complex the image is. A simple image with a clear subject gets a tighter budget, while a complex, busy scene is allowed to keep more tokens. This ensures that the system retains enough detail to handle difficult questions without wasting energy on simple ones.

Once this initial selection is made, the remaining tokens are passed to the large language model, which begins to process them alongside text. Here, the system employs a second pruning strategy that works like a pyramid. As the data moves through the deeper layers of the model, the system gradually removes more tokens. The logic is that as the model understands the image better in the context of the question, it becomes clearer which pieces of information are no longer needed. The researchers found that the most effective way to split the work between these two stages is not to divide the tokens evenly, but to follow a specific mathematical balance. They discovered that the ideal number of tokens to keep after the first stage is roughly the square root of the product of the starting number and the final target number. This rule of thumb, which they call an information-aware budget, allows the system to find a sweet spot where it saves enough computing power without sacrificing the ability to answer questions accurately.

To make sure the first stage doesn't accidentally delete tokens that the second stage would have found useful, the researchers added a third layer of intelligence. They analyzed how the visual tokens interact with each other inside the image encoder itself. They found that some tokens act as global integrators, connecting information across the whole image, while others are just local detectors. By giving extra weight to these global integrators during the first cut, the system ensures that the tokens passed to the language model form a high-quality foundation. This "cross-stage continuity" means that the second stage has a better starting point, allowing it to make smarter decisions about what to remove later.

The results of testing this framework on two major families of vision-language models were striking. When the researchers pushed the systems to extreme limits, keeping only a tiny fraction of the original tokens—sometimes as few as 24 out of 576—the new method significantly outperformed existing techniques. Single-stage methods, which try to do all the cutting at once, saw their accuracy drop sharply under these conditions. In contrast, the two-stage UniPrune approach maintained high performance, correctly answering questions about objects and scenes that other methods missed. For instance, on a standard test of visual understanding, the new method retained over 90% of the performance of the full, uncompressed model while using only a fraction of the computing power.

Beyond accuracy, the study also measured the practical benefits for running these models on real hardware. By reducing the number of tokens early in the process, the system drastically lowered the peak memory required to run the model. This is a critical factor for deploying these technologies on devices with limited resources. The researchers calculated that the new method reduced the total computational work by more than half compared to running the model without any pruning, and even more compared to methods that wait until the end to cut tokens. The overhead added by the new smart budgeting and scoring mechanisms was negligible, taking up less than one percent of the total time, which means the efficiency gains are almost entirely pure.

The study suggests that the key to efficient artificial intelligence is not just about cutting corners, but about understanding where and when to cut. By acknowledging that visual redundancy comes in two different forms—one based on the image itself and another based on the task at hand—the researchers created a system that handles both. The findings indicate that a unified, multi-stage approach is superior to trying to solve the problem in a single step. This work provides a clear path forward for making powerful vision-language models faster and more accessible, proving that with the right strategy, we can strip away the excess without losing the essence of what makes these systems intelligent.