⚡ electrical engineering

Measuring Cross-Jurisdictional Transfer of Medical Device Risk Concepts with Explainable AI

This study employs explainable AI to demonstrate that despite shared risk terminology, medical device classification logic exhibits sparse, asymmetric, and weak transferability across the US FDA, China NMPA, and EU MDR jurisdictions, challenging the assumption that common regulatory vocabulary implies portable classification mechanisms.

Original authors: Yu Han, Aaron Ceross

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yu Han, Aaron Ceross

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to sort medical devices (like pacemakers, bandages, or surgical tools) into "risk categories."

In the real world, three big bosses run the show: the USA (FDA), China (NMPA), and Europe (EU MDR). They all agree that some devices are "low risk" (like a bandage) and some are "high risk" (like a heart valve). They even use the same words to describe them, like "implantable" or "invasive."

The Big Question:
If we teach a robot the rules Europe uses to sort these devices, will that robot automatically become good at sorting devices for China and the USA? In other words, do these "risk concepts" travel easily across borders, or is every country's system totally different?

The Experiment: The "Universal Translator" Test

The researchers built a smart robot (an AI) and gave it a specific set of 7 "Risk Clues" derived from European rules. These clues are things like:

Is it implanted inside the body?
Is it invasive (cutting into the body)?
How long does it stay in the body?

They then asked the robot: "Can you use these 7 European clues to sort Chinese and American devices correctly?"

To make sure the test was fair, they ran it in two ways:

The "Clean" Test: They forced the robot to look at the device descriptions using the exact same simple method for all three countries (like reading a menu in three different languages with the same dictionary).
The "Local Expert" Test: They let the robot use the best, most specific tricks available for each country (like using a specialized translator for Chinese and a different one for English).

The Results: A Tale of Two Worlds

1. The "Clean" Test: The Signal is Lost 📉

When they used the fair, "clean" method, the result was almost zero.

The Analogy: Imagine trying to use a map of London to navigate the streets of Tokyo. Even though both cities have "streets" and "buildings," the map is useless because the layout is completely different.
The Finding: The European "Risk Clues" didn't help the robot sort Chinese or American devices at all. The shared vocabulary (words like "implantable") didn't translate into shared logic. The robot couldn't figure out that "implantable" meant "high risk" in the US or China just because it meant that in Europe.

2. The "Local Expert" Test: A Tiny Glimmer of Hope (But Mostly No) 🤏

When they let the robot use local tricks, there was a tiny improvement for China, but it was very small.

The Analogy: It's like giving the robot a cheat sheet that says, "In China, if the product code starts with '03', it's an implant." The robot got slightly better, but only because it was memorizing local codes, not because it truly understood the concept of risk.
The Finding: The improvement was so small that it might just be a fluke or a result of how the data was collected. For the US, the improvement was basically non-existent.

3. The "Class I" Disaster: The Empty Box Problem 📦

The biggest failure happened with Class I devices (the lowest risk, like bandages or tongue depressors).

The Analogy:
- Europe's Rule: "If a device doesn't have any 'dangerous' features (like being invasive), it's Class I." It's a Residual category (a "catch-all" for things that aren't dangerous).
- US/China Rule: "Class I is a specific list of things we know are safe." It's a Positional category (a specific box).
The Result: When the robot, trained on US/China lists, tried to guess European "catch-all" items, it failed miserably. It couldn't understand that "having no dangerous features" was a valid category. It kept trying to force them into higher-risk boxes.

Why Did This Happen? (The Metaphors)

The Language Barrier: The robot was trying to read Chinese descriptions, English descriptions, and European descriptions all at once. The computer code used to read the text couldn't understand that a Chinese word and an English word meant the same thing. It was like trying to solve a puzzle where the pieces are made of different materials.
The "Circular" Trap: For Europe, the researchers accidentally gave the robot the answer key. They derived the "Risk Clues" directly from the European classification rules. So, of course, the robot was great at sorting European devices! But this didn't prove it could sort other devices; it just proved it could memorize the European rules.
The US "Predicate" System: The US system is unique. They don't just look at the device; they ask, "Does an almost identical device already exist on the market?" The robot's "Risk Clues" (like "is it invasive?") couldn't capture this "history of similarity" logic. It's like trying to sort books by their cover color, but the US librarian sorts them by "which other book they look like."

The Bottom Line

Shared words do not mean shared logic.

Just because regulators in the US, China, and Europe all use the word "implantable," it doesn't mean they use that word to make decisions in the same way.

For AI Developers: You cannot just train an AI on one country's data and expect it to work in another. You have to test it, and you will likely find it fails.
For Regulators: We can't assume our systems are "harmonized" just because we use similar dictionaries. The underlying structures are too different.

In short: The "Universal Translator" for medical device risk is currently broken. We can't assume a concept that works in Europe will automatically work in China or the US. We need to build specific tools for each region rather than hoping for a one-size-fits-all solution.

1. Problem Statement

Medical device regulators in the United States (FDA), China (NMPA), and Europe (EU MDR) all utilize "risk" as a core concept for classification, yet they employ structurally distinct mechanisms:

EU MDR: Rule-based system (22 explicit rules in Annex VIII).
FDA: Predicate-based system (reliance on substantial equivalence to existing devices).
NMPA: Catalogue-based system (matching product descriptions to a classification catalogue).

The Core Question: Do shared regulatory vocabulary terms (e.g., "invasiveness," "implantability") carry portable classificatory signal across these different jurisdictions? In other words, can risk factors derived from one system's logic effectively predict classifications in another? Current AI regulatory tools often assume this portability without empirical validation.

2. Methodology

The authors treat Explainable AI (XAI) not just as a transparency tool, but as an empirical probe to measure regulatory structural overlap.

Dataset: 141,942 medical device records (53,745 FDA, 65,703 NMPA, 22,494 EU MDR).
Feature Engineering:
- Regulatory Factors: Seven specific risk factors were derived: implantability, duration of use, invasiveness, active device, sterile, measuring function, and reusable.
- Text Features: 42 dimensions derived from TF-IDF on character n-grams (to handle multilingual data: Chinese, English, French, German) via SVD.
Experimental Design:
- Ablation Study: Random Forest classifiers were trained under three conditions: (1) Text + Factors, (2) Text only, (3) Factors only.
- Metric: The primary metric was the change in F1-score ( $\Delta F1$ ) when adding regulatory factors to the text baseline.
- Two Extraction Pipelines:
  1. Symmetric Pipeline: Used uniform keyword matching across all jurisdictions to remove jurisdiction-specific extraction advantages (the "clean" signal).
  2. Jurisdiction-Specific Pipeline: Used the optimal extraction method for each region (e.g., rule-to-factor mapping for EU MDR, code-prefix for NMPA).
- Zero-Shot Transfer: Training on FDA/NMPA data and testing on EU MDR (and vice versa) without target-domain training examples.
- Perturbation Analysis: Searching for minimal factor changes required to reclassify a device to test decision boundaries.

3. Key Contributions

Empirical Measurement of Regulatory Overlap: Moves beyond qualitative comparison to quantitatively measure the "distance" between regulatory systems using AI performance metrics.
Demonstration of Asymmetry: Proves that regulatory concept transfer is not symmetric; concepts flow in specific directions (e.g., EU $\to$ China) but not others.
Methodological Framework for XAI in Regulation: Establishes a protocol for using concept-based XAI to test the validity of cross-jurisdictional AI assumptions rather than just explaining model outputs.
Identification of Structural Mismatches: Highlights the fundamental difference between "residual" class definitions (EU MDR Class I = absence of risk) and "positional" definitions (FDA/NMPA Class I = specific product codes).

4. Key Results

A. Factor Transferability ( $\Delta F1$ )

Symmetric Pipeline (Clean Signal): Under uniform extraction, factor contribution was negligible across all jurisdictions ( $\Delta F1 < 0.01$ ). This suggests that without pipeline-specific advantages, the shared vocabulary does not translate into shared classification logic.
Jurisdiction-Specific Pipeline:
- EU MDR $\to$ NMPA: Showed a modest gain ( $\Delta F1 = +0.024$ ). However, sensitivity analyses indicated this is an upper bound, conflating genuine concept transfer with extraction artifacts and device-category confounding.
- EU MDR $\to$ FDA: Negligible gain ( $\Delta F1 \approx +0.006$ ).
- Reverse Transfer: NMPA-derived factors did not improve EU MDR classification, and FDA-derived factors added almost no signal to other jurisdictions.
Conclusion: Cross-jurisdictional transfer is sparse, asymmetric, and weak.

B. Zero-Shot Transfer Failure

When training on FDA/NMPA and testing on EU MDR, the model achieved a macro-F1 of 0.384, but Class I collapsed to an F1 of 0.001.
Cause: EU MDR Class I is defined residually (devices that trigger no higher-risk rules), whereas FDA/NMPA Class I is defined positionally (specific catalogue entries). The model, trained on positional logic, cannot recognize the "absence of risk" pattern in the EU system.
Text Barrier: The TF-IDF text features failed to transfer across languages (Chinese vs. Latin scripts), forcing the model to rely almost entirely on the seven regulatory factors, which were insufficient to bridge the structural gap.

C. Sensitivity Analyses

Device-Type Control: Even when controlling for device categories, a residual signal remained for NMPA, suggesting some genuine regulatory overlap, though it was small.
Multilingual Embeddings: Replacing TF-IDF with multilingual sentence embeddings (MiniLM) improved zero-shot performance slightly but did not resolve the Class I collapse, confirming the failure is structural, not just linguistic.

5. Significance and Implications

Challenging Assumptions: The study refutes the common assumption in regulatory AI that shared vocabulary implies shared logic. "Invasiveness" in the EU does not functionally equate to "invasiveness" in the US or China for classification purposes.
Policy & Harmonization: Efforts to harmonize regulations (e.g., via IMDRF) face deep structural hurdles. AI models cannot simply be "ported" between jurisdictions; they require explicit, jurisdiction-specific empirical validation.
Role of XAI: The paper demonstrates that XAI can be used as a diagnostic tool to measure the limits of model transferability, revealing where regulatory systems diverge fundamentally.
Practical Guidance: Developers of cross-border regulatory AI must account for:
1. Extraction Asymmetry: How factors are derived matters as much as the factors themselves.
2. Class Definition Mismatch: Residual vs. positional class definitions create insurmountable barriers for zero-shot transfer.
3. Language Barriers: Text representations must be cross-lingually aligned to avoid treating language as a proxy for jurisdiction.

Final Verdict: Shared regulatory vocabulary does not translate into portable classification logic. Cross-jurisdictional AI deployment requires measured portability rather than assumptions of structural compatibility.