Technical Summary: Kernel Alignment-based Hybrid Heterogeneous Attribute Space Three-way Clustering for Disease Feature Recognition

Problem Statement

The integration of heterogeneous medical data (numerical, categorical, and time-series) for disease diagnosis faces significant challenges in feature selection. Existing methods struggle with three primary limitations:

1. Measurement Distortion: Traditional similarity metrics (e.g., Euclidean distance, simple concatenation) fail to provide a unified, interpretable framework for measuring similarity across mixed modalities, often leading to biases in clinical decision-making.
2. Inadequate Uncertainty Modeling: Classical feature selection algorithms rely on rigid, binary partitioning (hard thresholds). This approach fails to capture the inherent boundary fuzziness and fuzzy correlations among medical attributes, increasing the risk of erroneously eliminating critical pathogenic factors or retaining noise.
3. Insufficient Feature Optimization: Existing kernel-based methods often lack a synergistic mechanism for the dynamic, iterative optimization of attribute redundancy and cluster centers, making it difficult to identify low-redundancy feature subsets in high-dimensional, noisy medical environments.

Methodology

The paper proposes a novel feature selection framework that integrates Kernel Alignment with Three-way Clustering in the attribute space. The methodology proceeds through the following stages:

1. Unified Kernel Alignment Metric Framework

To address the heterogeneity of data types, the authors construct a unified metric space using specific kernel functions mapped to a Reproducing Kernel Hilbert Space (RKHS):

• Numerical Attributes: Mapped using Adaptive Gaussian Kernels, which adjust bandwidth based on local sample density to avoid over-smoothing or over-sensitivity.
• Categorical/Ordinal Attributes: Mapped using Ordinal Matching Kernels, which measure absolute rank differences for ordered variables (e.g., disease severity) and map distances to the interval [0, 1].
• Time-Series Attributes: Mapped using Spectral Domain Kernels, which extract power spectral density features (via Welch method) to capture frequency-domain similarities, mitigating issues with phase shifts and non-stationarity in signals like ECG.

These kernels generate kernel matrices for each attribute. The Kernel Alignment theory is then employed to quantify the consistency between an attribute's kernel matrix and an ideal target kernel matrix (derived from disease labels). This provides a unified measure of both feature redundancy (similarity between attributes) and feature importance (inconsistency with the decision attribute).

2. Attribute Space Three-way Clustering

The framework extends three-way clustering from the sample space to the attribute space. Instead of hard partitioning, attributes are divided into three regions based on evaluation functions and thresholds $(\alpha, \beta)$:

• Core Region ($Co$): Attributes with high similarity to a cluster center (definite membership).
• Fringe/Boundary Region ($Fr$): Attributes with ambiguous similarity (deferred decision).
• Trivial Region ($Tr$): Attributes with low similarity (definite non-membership).

This "deferment" strategy explicitly models boundary uncertainty, allowing the algorithm to handle noisy features without immediate elimination.

3. Adaptive Feature Selection via Joint Loss Function

The algorithm operates in two stages to optimize the feature subset:

• Initial Generation: An adaptive mechanism generates initial cluster centers based on inconsistency measures with the decision attribute, without requiring preset cluster numbers.

• Global Optimization: A Joint Loss Function is formulated to dynamically optimize cluster centers. This function balances two objectives:

  1. Decision Relevance: Minimizing inconsistency with the target disease label.
  2. Inter-cluster Redundancy: Maximizing independence between cluster centers.

  The loss function is defined as: $Loss(a) = \lambda \cdot Incons(a, D) + (1 - \lambda) \cdot Red(a, C \setminus \{C_k\})$, where $\lambda$ controls the trade-off. The algorithm iteratively selects the optimal center for each cluster to minimize this loss.

Key Contributions

1. Unified Heterogeneous Metric: The proposal of a unified kernel alignment framework that effectively maps numerical, categorical, and temporal data into a single RKHS, overcoming the limitations of linear methods in handling complex nonlinear interactions.
2. Attribute Space Three-way Clustering: The extension of three-way clustering to the feature space, utilizing a "deferment" mechanism to explicitly characterize boundary uncertainty and fuzzy correlations among heterogeneous features, thereby reducing the risk of discarding critical pathogenic factors.
3. Joint Loss Optimization: The development of a joint loss function that integrates decision relevance and inter-cluster redundancy, enabling the adaptive reduction of feature subsets through dynamic optimization of cluster centers.

Experimental Results

The method was validated on the Symile-MIMIC multimodal clinical dataset, focusing on pulmonary disease severity prediction. The dataset included 54 mixed heterogeneous features (patient demographics, 50 blood lab indicators, and 12-lead ECG time-series) across 6,654 samples.

• Dimensionality Reduction: The method successfully compressed the feature dimension from 54 to 15 (a 72% reduction).
• Classification Performance: Compared to the original unselected feature set, the proposed method improved classification accuracy across four mainstream classifiers:
  • SVM: +6.1%
  • KNN: +6.6%
  • XGBoost: +5.0% (reaching 66.27% accuracy)
  • Random Forest: +4.7%
• Comparison with Baselines: The proposed method outperformed traditional similarity metrics (Cosine, Pearson, Mutual Information, Distance Correlation) and other feature selection baselines. Notably, it achieved superior performance in handling the high-dimensional ECG data without the information loss associated with forced dimensionality reduction (e.g., PCA) required by other methods.
• Stability: The selected feature subset demonstrated high distributional stability, with a median Kuncheva Stability Index of approximately 0.68.
• Early Warning Capability: In handling highly imbalanced long-tailed data, the subset combined with XGBoost achieved an AUC of 0.622 for early disease warning (Severity $\ge$ 1).

Significance and Claims

The paper claims that the integration of kernel-based nonlinear mapping with the three-way "deferment" strategy significantly enhances the robustness of feature extraction in noisy medical environments. By moving beyond rigid binary partitioning, the framework effectively manages boundary fuzziness, ensuring that critical pathogenic factors are not erroneously eliminated.

The authors assert that their method provides interpretable support for multimodal medical decision-making by identifying synergistic pathogenic factors (e.g., the correlation between ECG signals and immune-inflammatory markers like eosinophil counts) that align with established clinical knowledge. The study concludes that this approach offers a superior balance between dimensionality reduction and predictive performance compared to traditional similarity-based methods, particularly for complex, mixed-modality clinical data.

Limitations Acknowledged by the Authors:

• The current framework does not systematically incorporate unstructured medical imaging (e.g., chest X-rays) into the kernel alignment system.
• The iterative optimization of three-way clustering increases time complexity for large-scale datasets.
• Validation is currently limited to the Symile-MIMIC dataset due to the scarcity of open-source multimodal datasets with strict spatio-temporal alignment.
• Direct benchmarking against classical algorithms (e.g., mRMR, LASSO) is structurally difficult because those methods often require flattening temporal signals, which destroys the topological dependencies preserved by the proposed kernel framework.