🤖 AI

ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression

ORCA is a training-free, plug-and-play method for 3D CT visual token compression that leverages organ-guided aggregation and centroid sinusoidal encoding to preserve anatomical information, significantly reducing token counts and inference costs while outperforming existing baselines across attribute prediction and text generation tasks.

Original authors: Renjie Liang, Zijian Xu, Jinqian Pan, Chengkun Sun, Zhengkang Fan, Shawn Li, You Qin, Mei Liu, Jie Xu

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Renjie Liang, Zijian Xu, Jinqian Pan, Chengkun Sun, Zhengkang Fan, Shawn Li, You Qin, Mei Liu, Jie Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to read a book, but instead of words, the book is made of millions of tiny, glowing 3D blocks. This is the world of 3D CT scans, which are like taking a thousand X-ray slices of a human body and stacking them to see inside without cutting anyone open. To help a robot understand these scans, scientists use Vision-Language Models (VLMs). Think of these models as a team: a "vision expert" that looks at the blocks and a "language expert" (a large language model) that writes reports or answers questions.

The problem is that a single CT scan creates a massive pile of these blocks—sometimes tens of thousands of them. If you try to feed all of them to the language expert at once, it gets overwhelmed, like trying to drink from a firehose. The computer runs out of memory, or the process takes forever. So, scientists have to compress the data, shrinking that huge pile down to a manageable handful of "summary tokens" before the language expert sees them. The big question is: How do you shrink a 3D picture without throwing away the important details? If you just squish the blocks together randomly, you might mix a healthy lung with a tiny tumor, and the robot will never find the disease. This paper tackles that exact puzzle.

The Problem: The "Smoothie" Mistake

Imagine you have a giant, delicious fruit salad in a 3D box. You have a tiny robot that needs to describe the fruit, but it can only hold a few handfuls of fruit at a time. The old way of doing this, called Grid Average, is like taking a rigid grid and forcing the robot to scoop up whatever falls into each square.

If a square happens to catch a strawberry, a piece of broccoli, and a chunk of air all at once, the robot has to blend them into a single "smoothie" token. The result? A brownish mush that tastes like nothing. In medical terms, this means a tiny, dangerous nodule (the strawberry) gets blended into the surrounding healthy tissue (the broccoli) and the empty space (the air). The robot loses the nodule entirely. It's like trying to find a needle in a haystack by turning the haystack into a smoothie; the needle is still there, but you can't see it anymore.

Other methods try to be smarter by picking only the "important" blocks, but they often require the robot to guess which ones are important based on complex rules or specific questions, which doesn't always work well for every situation.

The Solution: ORCA, the Organ-Savvy Organizer

Enter ORCA (ORgan-Centroid Aggregation). The authors of this paper built a new, training-free tool that acts like a super-organized librarian for your 3D fruit salad. Instead of using a rigid grid or guessing which blocks to keep, ORCA does two clever things:

It groups by "neighborhood" and "organ."
ORCA looks at the blocks and says, "Hey, these blocks look like they belong to the same organ, and they are right next to each other. Let's keep them together." It merges adjacent blocks that are similar, but it uses a secret map (called an organ mask) to make sure it doesn't accidentally glue a heart block to a lung block. It's like grouping all the strawberries together in one basket and all the broccoli in another, rather than forcing them into a random grid square. This ensures that a tiny nodule stays with its own kind of tissue and doesn't get blended away.
It leaves a "GPS tag" on every group.
When you merge blocks into a group, you lose the exact map of where that group was in the original 3D box. If you just hand the robot a basket of strawberries, it doesn't know if they were in the top-left corner or the bottom-right. ORCA solves this by attaching a special sinusoidal encoding (a fancy mathematical way of saying "GPS coordinates") to every group. It tells the robot exactly where the center of that group was in the original scan. So, even though the robot sees a smaller pile of tokens, it knows exactly where to look in the 3D space.

What They Found: Smarter, Faster, and Training-Free

The researchers tested ORCA on two different types of CT scans (chest and abdomen) and used five different "vision experts" to see if it worked with everyone. They compared it against the old "Grid Average" method and other fancy compression tricks.

Here is what they discovered:

It keeps the details: At the same number of tokens, ORCA was much better at preserving specific details like location (where an organ is), size, and density (how heavy the tissue is). For example, when predicting the location of an organ, ORCA was significantly more accurate than the old methods, which often got lost.
It's a plug-and-play tool: The best part? ORCA doesn't need to be trained. You don't have to feed it thousands of examples to learn how to work. You just drop it into the pipeline, and it works immediately. It's a "drop-in" replacement for the old methods.
It saves massive resources: By compressing the data, ORCA shrinks the amount of information the computer has to process by 64 times. This makes the computer 31 times faster and reduces the memory needed for the "thinking" part (the KV-cache) by 50 times.
It works for reports and questions: Whether the robot is answering specific questions (like "How big is the heart?") or writing a full medical report, ORCA helped it perform better than the other methods, especially when the data was heavily compressed.

The Bottom Line

The paper argues that the old way of squashing 3D CT scans into a grid is too blunt; it mixes up important details. ORCA offers a smarter, training-free way to compress these scans by grouping similar, connected parts and tagging them with their location. The authors show that this method preserves the crucial anatomical information a robot needs to make accurate diagnoses, all while making the process much faster and cheaper. It's a simple but powerful idea: don't just shrink the picture; organize it so the robot can still find the needle in the haystack.

Technical Summary: ORCA – ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression

Problem Statement

3D Computed Tomography (CT) scans generate dense volumetric data that, when processed by Vision-Language Models (VLMs), result in extremely long sequences of visual tokens (often thousands to tens of thousands per volume). Before a Large Language Model (LLM) can consume this data, the token sequence must be compressed.

Existing compression strategies face significant limitations in the 3D medical domain:

Grid Average Pooling: The common baseline pools tokens over regular 3D grid cells. While simple, it is "anatomy-blind," often blending distinct anatomical structures (e.g., organs, lesions, vessels, and air) into a single token, thereby erasing critical diagnostic evidence like small nodules.
Token Pruning: Methods that discard tokens based on attention scores or text queries make irreversible hard decisions. Once a region is pruned, its evidence cannot be recovered, and many methods require specific model architectures or text inputs to function.
Integrated Encoders: Some approaches build compact representations directly into the encoder. However, these are tied to specific training objectives and architectures, making them non-modular and unable to serve as a standalone compressor for arbitrary token budgets.

There is a critical gap for a training-free, plug-and-play compressor that reduces token counts while preserving the specific anatomical evidence required for downstream tasks like report generation and visual question answering (VQA).

Methodology: ORCA

The authors propose ORCA (ORgan-Centroid Aggregation), a training-free token compressor designed specifically for 3D CT visual tokens. ORCA operates at the interface between a frozen 3D encoder and the downstream projector/LLM.

Core Mechanisms:

Connected Region Aggregation:
- ORCA treats the initial token grid as a set of individual regions.
- It iteratively merges adjacent regions (sharing a face in the 3D grid) using Ward linkage to minimize within-region distortion.
- Organ Guidance: To prevent mixing unrelated anatomy, the merge cost is augmented with soft organ-mask guidance. For each token, an organ-coverage vector (derived from tools like TotalSegmentator) is scaled and concatenated with the visual embedding. Regions with similar visual features and similar organ coverage are favored for merging. This ensures that merged regions remain spatially connected and anatomically coherent without forcing strict adherence to predefined organ masks.
- The process stops when the target token budget ( $B$ ) is reached.
Centroid Position Encoding:
- Merging tokens destroys the original regular grid structure, meaning the sequence order no longer reliably indicates spatial location.
- To restore spatial layout, ORCA computes the 3D centroid of each merged region.
- A sinusoidal encoding of this centroid is appended to the region's averaged visual embedding. This explicit position signal allows the downstream model to localize regions regardless of their shape or size.
Training-Free and Plug-and-Play:
- ORCA requires no model surgery, attention hooks, additional supervision, or changes to the downstream training recipe. It functions as a drop-in replacement for standard pooling layers.

Key Contributions

Novel Compressor: Introduction of ORCA, a training-free method that aggregates spatially connected, feature-similar regions using organ guidance and restores spatial layout via sinusoidal centroid encoding.
Comprehensive Evaluation: Evaluation across two datasets (CT-RATE for chest CT, Merlin for abdomen CT) and five different encoders (including COLIPRI, CT-CLIP, BTB3D, SuPreM, and SegVol).
Task Diversity: Assessment across two task types:
- Attribute Prediction: Probing for five attribute families (disease, size, density, location, texture).
- Text Generation: Visual Question Answering (VQA) and radiology report generation.
Role Separation: Demonstration that the encoder determines the capacity of information in the tokens, while the compressor governs how much of that information survives to the downstream model.

Experimental Results

1. Attribute Probing (Representation Quality):

Location Preservation: ORCA significantly outperforms baselines (Grid Average, ToMe, MedPruner-DINS) in preserving spatial location information. On COLIPRI at $B=216$ , ORCA achieves an $R^2$ of 0.677 for location, compared to 0.247 for Grid Average. This is attributed to the explicit centroid encoding.
Anatomical Fidelity: ORCA consistently matches or exceeds the performance of other compressors across size, density, and texture attributes. It preserves more information than Grid Average, which tends to blur distinct anatomical features.
Robustness: ORCA's advantages persist across different encoder architectures (language-supervised vs. reconstruction-based) and anatomical domains (chest vs. abdomen).

2. Text Generation (LLM Performance):

Report Generation: ORCA leads in clinical efficacy metrics (GREEN score), particularly under heavy compression ( $B=27$ ). It outperforms Grid Average and MedPruner-DINS, indicating that the preserved anatomical details translate to better clinical reporting.
Measurement VQA: ORCA achieves the highest accuracy in answering questions regarding size, density, and location, with the largest margin of improvement in location-based queries.
Efficiency: ORCA reduces the visual context by 64× and the KV-cache size by 50×. It reduces prefill latency by 31× compared to uncompressed baselines, while end-to-end report time remains largely unchanged as these short runs are decode-dominated.

3. Ablation Studies:

Position Encoding: Removing the centroid encoding drops location performance to the level of Grid Average (~0.25), confirming that explicit position injection is critical.
Organ Guidance: While the organ mask improves performance, the method remains robust even without it, though the mask helps recover content that Grid Average blends away (e.g., density metrics improved from 0.865 to 0.913).
Disease Saturation: For encoders pre-trained on radiology reports (COLIPRI, CT-CLIP), the "disease" attribute is often saturated, meaning compression has little room to improve it. However, for reconstruction-based encoders (BTB3D), ORCA recovers significant disease signal (+0.097 improvement), demonstrating its ability to recover information lost by standard pooling.

Significance and Claims

The paper claims that ORCA addresses a fundamental bottleneck in 3D CT VLMs: the loss of anatomical evidence during token compression. By shifting the compression paradigm from fixed-grid pooling to content-adaptive, anatomy-guided aggregation with explicit position re-injection, ORCA ensures that downstream models retain the spatial and structural details necessary for clinical tasks.

The authors emphasize that ORCA is a drop-in replacement that requires no retraining of the encoder or the LLM. It demonstrates that a simple, training-free operator can preserve most of the recoverable information at a fraction of the token cost (8× to 64× reduction), making high-fidelity 3D CT analysis feasible for resource-constrained VLMs. The work suggests that for 3D medical imaging, the "compressor" plays a distinct and vital role in determining which anatomical evidence survives to the decision-making stage.