Managing Map Cardinality in Automatic Disease Classification Mapping: Balancing Precision, Recall and Coverage

Technical Summary: Managing Map Cardinality in Automatic Disease Classification

Problem Statement
The automatic mapping of disease classification systems, such as the International Classification of Diseases (ICD), is critical for health data integration and longitudinal analysis. However, existing embedding-based methods primarily focus on one-to-one mappings, failing to address complex one-to-many scenarios where a single source code semantically corresponds to multiple target codes. Current selection strategies to handle cardinality—specifically threshold-based and top-K methods—suffer from inherent trade-offs between precision, recall, and mapping coverage (the proportion of source codes with at least one mapping).

• Threshold-based methods prioritize precision at the expense of recall and coverage; high thresholds yield low coverage, leaving many source codes unmapped.
• Top-K methods guarantee high coverage and recall but sacrifice precision by forcing a fixed number of matches, often introducing significant false positives (e.g., generating $K-1$ false positives for valid one-to-one mappings).

These limitations create a bottleneck for manual validation, where terminology experts must either sift through excessive false positives or manually map large gaps of unmapped data.

Methodology
The authors propose a novel blocking-and-matching framework, inspired by entity resolution pipelines, to automatically generate mappings between different ICD versions (e.g., ICD-9-CM $\leftrightarrow$ ICD-10-CM and ICD-10-AM $\leftrightarrow$ ICD-11). The approach operates in two distinct stages:

1. Blocking Stage (Candidate Generation):
   Instead of evaluating all possible pairs, the method generates a reduced set of candidate target codes for each source code. The authors introduce a hybrid selection strategy combining:

   • Top-K Selection: Retrieves the top $K$ target codes based on embedding similarity.
   • Bidirectional Mapping Expansion (BiMaps): Constructs one-to-one mappings in both directions (Source $\to$ Target and Target $\to$ Source) and aggregates them.
     The union of these sets forms the "block" of candidates. This hybrid approach aims to maximize recall (retaining true matches) while minimizing block size to avoid the "lost in the middle" phenomenon in Large Language Models (LLMs), where excessive context degrades performance.

2. Matching Stage (LLM Selection):
   The matching task is reformulated as a Multiple-Choice Question (MCQ). For each source code and its block of candidates, a prompt is constructed instructing a Large Language Model (specifically Qwen3-8B) to identify all semantically equivalent target codes. Unlike traditional Entity Resolution which often selects a single match, this approach allows the LLM to select zero, one, or multiple options, directly addressing the one-to-many cardinality challenge.

Key Contributions

1. Novel Framework: Introduction of a blocking-and-matching pipeline that integrates embedding-based candidate generation with LLM-driven selection for ICD mapping.
2. Trade-off Mitigation: The method addresses the precision-recall-coverage trade-off inherent in selection-based methods, delivering high precision while maintaining comparable recall and broad mapping coverage.
3. Hybrid Blocking Strategy: The development of a Top-K + BiMaps strategy that balances computational efficiency with the retention of valid candidate pairs.
4. Empirical Validation: Demonstration of balanced performance across multiple ICD version pairs and three specific disease chapters (Digestive, Infectious/Parasitic, and Respiratory).

Results
The study evaluated the proposed method against baseline threshold-based ($\lambda=0.85$) and top-K ($K=5$) methods across four mapping tasks.

• Precision: The blocking-and-matching framework achieved the highest precision across nearly all tasks (ranging from 0.47 to 0.71), representing a 2–3.5x improvement over the Top-K baseline and a 5–35 percentage point improvement over BiMaps in most cases.
• Recall: The framework maintained competitive recall (0.53–0.85), comparable to the high-recall BiMaps and Top-K baselines, without the precision penalties associated with indiscriminate Top-K selection.
• Coverage: The method achieved near-perfect mapping coverage (MC $\ge$ 0.98), ensuring almost every source code received at least one candidate mapping, thereby overcoming the coverage limitations of threshold-based methods.
• F1-Score: The framework achieved the highest F1-scores (0.53–0.77) in 11 out of 12 mapping tasks, demonstrating a superior balance between precision and recall.

Significance and Claims
The paper claims that the proposed framework offers a robust, training-free solution for automatic disease classification mapping. By leveraging the general-purpose medical knowledge embedded in pre-trained LLMs via zero-shot prompting, the method avoids the need for task-specific fine-tuning or labeled mapping data.

The authors emphasize that while the LLM-based approach introduces computational overhead (e.g., ~43 hours for the evaluated datasets), this is a justifiable upfront investment compared to the labor-intensive, time-consuming nature of manual mapping, which often requires large teams of experts and significant funding. The method is particularly significant for major ICD version transitions (occurring infrequently), where creating high-quality, reusable mapping tables is essential for downstream health data interoperability. The approach effectively reduces the manual workload for terminology experts by filtering out irrelevant candidates (high precision) while ensuring no valid mappings are missed (high coverage).