💬 NLP

Explaining BiomedCLIP with Weighted Banzhaf Interactions Supported by Tree-Gram Parsing

This paper introduces ParseFIxLIP, a novel explanation framework that integrates Tree-Gram Parsing with Weighted Banzhaf interactions to group fragmented text tokens into semantically coherent medical concepts, thereby enhancing the interpretability and robustness of BiomedCLIP's decision-making in clinical settings.

Original authors: Jakub Rymarski (University of Warsaw, Poland), Adam Rempała (University of Warsaw, Poland), Bartłomiej Sobieski (University of Warsaw, Poland), Przemysław Biecek (University of Warsaw, Poland)

Published 2026-07-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jakub Rymarski (University of Warsaw, Poland), Adam Rempała (University of Warsaw, Poland), Bartłomiej Sobieski (University of Warsaw, Poland), Przemysław Biecek (University of Warsaw, Poland)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to understand the world by showing it pictures and reading stories to it at the same time. This is the world of Vision-Language Models (VLMs), a branch of artificial intelligence that acts like a digital detective, connecting what it sees in an image with what it reads in a caption. In the medical field, these robots are being trained to look at X-rays and CT scans and read the doctors' reports to help diagnose patients. But here's the catch: these robots are often "black boxes." They give an answer, but they don't explain why. If a robot says, "This X-ray shows a broken bone," a doctor needs to know exactly which part of the image and which specific words in the report led to that conclusion. Without a clear explanation, doctors can't trust the robot in life-or-death situations.

To solve this mystery, scientists use a mathematical tool called Game Theory. Think of the AI's decision-making process as a team game where every single piece of information (a pixel in the image or a tiny chunk of a word) is a "player." The game asks: "If we remove this player, does the team's score drop?" By playing this game over and over, we can figure out which players are the MVPs and which are just sitting on the bench. However, there's a major problem with how these games are currently played in medicine. The AI doesn't read words like "heart" or "fracture" as whole units. Instead, it chops them up into tiny, meaningless fragments (like breaking "heart" into "he" and "art"). This is like trying to solve a jigsaw puzzle where someone has cut the pieces into dust before you even started. The result is a chaotic, confusing explanation that no human can understand.

This is where a new study by Jakub Rymarski and his team comes in. They are tackling the problem of how to make these AI explanations clear and trustworthy for doctors. They introduce a clever new method called ParseFIxLIP. Instead of letting the AI play the game with those tiny, broken word fragments, they use a linguistic map (called a dependency parser) to glue the fragments back together into meaningful chunks before the game starts. Imagine taking those dust particles and gluing them back into whole words, and then gluing those words into full phrases like "saddle embolus" or "right kidney." By doing this, the game becomes much simpler and the answers become much clearer.

The team tested their idea on a medical AI called BiomedCLIP using a dataset of radiology images and reports called ROCOv2. They compared their new "glued-together" method against the old "broken-fragments" method. The results were striking. When the medical reports were short, both methods worked okay, but the new method was more stable. However, when the reports got long and complex (over 30 words), the old method completely fell apart. It tried to juggle too many tiny pieces at once, leading to a "curse of dimensionality" where the math got so messy that the explanation became useless (even showing negative scores, which means the explanation was worse than random guessing). In contrast, the new ParseFIxLIP method kept its cool. By grouping words into smart, semantic units (like treating "saddle embolus" as one single player instead of four broken ones), it reduced the complexity of the game. This allowed the AI to give a clear, statistically robust explanation even for long, complicated medical reports.

The researchers also ran some fun "stress tests" to see how the AI really thinks. They found that the AI is surprisingly fragile. If you rotate a medical image upside down, the AI gets confused and stops recognizing the anatomy, even though the shape is the same. It also tends to "cheat" by focusing too much on obvious markers like red arrows or text labels on the image, rather than the actual medical condition. ParseFIxLIP was able to reveal these quirks clearly, showing exactly where the AI was looking and what it was ignoring.

In short, this paper suggests that by using the structure of language to group words together before asking the AI to explain itself, we can get much better, more reliable answers. It doesn't just make the math work better; it makes the AI's reasoning look more like how a human doctor thinks—by understanding whole concepts rather than scattered fragments. While the method isn't a magic fix for every problem (it still relies on the underlying language tools, which can sometimes make mistakes), it offers a much clearer window into how these powerful medical AIs are making their decisions, which is a huge step toward trusting them in real hospitals.

Technical Summary: ParseFIxLIP for BiomedCLIP Interpretability

Problem Statement
Vision-Language Models (VLMs), such as BiomedCLIP, have demonstrated strong zero-shot capabilities in medical tasks like radiology analysis. However, their deployment in clinical settings is hindered by a lack of faithful, interpretable explanations. Existing game-theoretic explanation frameworks, specifically FIxLIP, struggle when applied to modern VLMs due to the reliance on subword tokenizers (e.g., BPE, WordPiece).

In the medical domain, this tokenization approach causes concept fragmentation. Precise clinical terms (e.g., "saddle embolus") are split into arbitrary, semantically meaningless subwords (e.g., "sad", "dle", "embol", "us"). This fragmentation leads to two critical issues:

Semantic Incoherence: Explanations become unreadable for clinicians, as importance is distributed across scattered fragments rather than unified medical concepts.
Statistical Instability: The fragmentation creates a high-dimensional feature space with severe multicollinearity. When analyzing long radiology reports, the combinatorial explosion of possible token masks triggers the "curse of dimensionality," causing standard interaction estimation methods to fail in isolating meaningful signals from noise, often resulting in poor goodness-of-fit metrics (e.g., negative Adjusted $R^2$ ).

Methodology: ParseFIxLIP
To address these challenges, the authors introduce ParseFIxLIP, a model-agnostic extension of the FIxLIP framework that integrates Tree-Gram Parsing into the Weighted Banzhaf interaction game.

Instead of treating raw subword tokens as individual players in the cooperative game, ParseFIxLIP projects the explanation task into a reduced-dimensional space using a surjective mapping function $\pi: N \to N'$ . This mapping groups raw tokens into semantically coherent "macro-players" based on dependency parsing trees generated by spaCy. The authors propose four grouping strategies:

Raw: The baseline identity mapping (standard tokenization).
Words: Merging subwords into full dictionary words.
Noun_chunks: Grouping multi-word noun phrases into single entities.
Smart_depth: A novel syntactic strategy that recursively groups dependency tree heads with their modifiers, ensuring complex medical concepts are unified.

Theoretically, this formulation defines a quotient game $\mathcal{G}' = (N', \nu')$ . By reducing the number of players ( $|N'| \ll |N|$ ) while maintaining semantic parsimony, the method structurally eliminates noise caused by subword fragmentation before interactions are computed. This reduction stabilizes the Weighted Banzhaf estimation under a fixed sampling budget, mitigating multicollinearity and the curse of dimensionality.

Key Contributions
The paper advances the state of VLM interpretability through three primary contributions:

Linguistically-Informed Game-Theoretic Explanation: ParseFIxLIP is presented as the first work to address token fragmentation in game-theoretic XAI by using dependency parsing trees to group tokens into coherent clinical concepts.
Theoretical Mitigation of Dimensionality: The quotient game formulation is shown to inherently solve multicollinearity and high-dimensionality issues, stabilizing the estimation of Weighted Banzhaf values.
Empirical Validation on Medical Data: The approach is evaluated on the ROCOv2 dataset, demonstrating that semantic grouping maintains statistical robustness where standard baselines fail, particularly on long radiology captions.

Experimental Results
The authors evaluated the method using Spearman's Rank Correlation and Adjusted $R^2$ as faithfulness metrics, alongside Insertion/Deletion curves (Area Between Curves - AID).

Quantitative Findings:
- On short captions, semantic grouping strategies (noun_chunks and smart_depth) achieved median Adjusted $R^2$ values above 0.7, significantly outperforming raw token baselines.
- On long captions (>30 words), the raw token baseline suffered from the curse of dimensionality, yielding negative Adjusted $R^2$ values for some samples due to overfitting on noisy, high-dimensional token sets. In contrast, smart_depth maintained statistical robustness by condensing inputs into fewer than 15 semantic concepts.
- While raw baselines occasionally showed slightly higher AID scores on long texts (by tracking scattered attention to background noise), semantic grouping provided more clinically relevant signals.
Qualitative Findings:
- Concept Unification: Visualizations confirmed that smart_depth successfully unified fragmented terms like "saddle embolus" and "left parotiditis" into single, high-importance concepts, whereas raw methods distributed importance across meaningless fragments.
- Robustness Testing: Stress tests revealed that BiomedCLIP itself lacks robustness to spatial transformations (e.g., upside-down rotation) and relies heavily on explicit annotations (arrows, text) rather than pathology. ParseFIxLIP faithfully captured these limitations, showing that the model's attention shifts to transformation-robust artifacts (like printed numbers) when anatomy is rotated, and disproportionately weights explicit markers over the actual disease.

Significance and Claims
The paper claims that ParseFIxLIP offers a critical step toward the responsible deployment of VLMs in medicine by providing intuitive and clinically relevant insights. By unifying complex medical concepts, the method fulfills the need for coherent explanations that clinicians can trust.

Crucially, the work does not merely improve explanation quality; it serves as a diagnostic tool that exposes fundamental limitations in current VLMs. The authors demonstrate that BiomedCLIP is fragile to spatial transformations and prone to "shortcut learning" via visual annotations. By revealing these behaviors through coherent, dependency-based grouping, ParseFIxLIP highlights the necessity for extreme caution when interpreting BiomedCLIP outputs in clinical decision-making. The approach successfully bridges the gap between mathematical model inputs and human-understandable medical concepts without requiring auxiliary models or human-annotated concepts.

Technical Summary: ParseFIxLIP for BiomedCLIP Interpretability

More like this