Learning predictive models for combinations of heterogeneous proteomic data sources

Technical Summary: Learning Predictive Models for Combinations of Heterogeneous Proteomic Data Sources

Problem Statement
The proliferation of technologies for measuring protein expression levels (e.g., in blood or urine) offers significant potential for disease detection. However, researchers face a critical challenge in understanding how to effectively utilize data generated by multiple, heterogeneous technologies simultaneously. Specifically, this work addresses the difficulty of combining two distinct proteomic data sources: Luminex multiplexed protein arrays and SELDI-TOF-MS (Surface-Enhanced Laser Desorption/Ionization Time-of-Flight Mass Spectrometry) profiles.

The core problem is that these data sources possess fundamentally different characteristics:

1. Biological Independence: Luminex measures specific, pre-selected immunogenic and growth factor markers, while SELDI-TOF-MS captures thousands of mass measurements representing a broader, less specific profile.
2. Dimensionality and Correlation: SELDI-TOF-MS data is high-dimensional (hundreds of measurements) with significant signal correlation, whereas Luminex data is lower-dimensional (typically 30–60 probes) with largely independent measurements.
3. Model Failure: The authors demonstrate that standard machine learning approaches, which work well on individual datasets, often fail or degrade in performance when the raw data from these two sources is naively merged into a single dataset for a single classifier.

Methodology
The study utilizes a pancreatic cancer dataset comprising 106 samples (56 cases and 53 matched controls) from the University of Pittsburgh Cancer Institute. The data includes 1,554 SELDI-TOF-MS peaks and 30 Luminex measurements.

• Preprocessing: SELDI-TOF-MS data underwent five preprocessing steps (variance stabilization, baseline correction, smoothing, intensity normalization, and profile alignment) using the Proteomic Data Analysis Package (PDAP). Peak selection was performed based on mean profiles and signal derivatives.
• Base Classifiers: The authors evaluated Support Vector Machines (SVM) and Random Forests (RF), chosen for their robustness in high-dimensional, small-sample scenarios.
• Combination Strategies:
  1. Data Fusion (Naïve): Merging raw features from both sources and training a single classifier.
  2. Model Fusion (Proposed): Instead of merging data, the authors propose combining the outputs of classifiers trained on individual data sources. Two specific approaches were investigated:
     • Model Inclusion: Using the soft output (e.g., distance from hyperplane for SVM, or tree ratio for RF) of a classifier trained on one dataset as an additional feature for a classifier trained on the other dataset.
     • Model Composition: A two-layer architecture where base classifiers generate soft discriminative projections (e.g., SVM distance and RF tree ratio), which serve as inputs for a second-level classifier (e.g., Naïve Bayes).

Key Results

• Individual Performance: On individual datasets, Random Forests performed best on Luminex data (AUC 0.98), while SVMs performed best on SELDI-TOF-MS data (AUC 0.90).
• Failure of Data Fusion: When the raw data was merged, performance deteriorated significantly. For instance, the Random Forest error rate increased from 9.41% (Luminex only) to 32.35% (merged), and the SVM error rate increased from 23.97% (MS only) to 34.49% (merged). The authors attribute this to the high dimensionality of MS data overwhelming the RF and the linear nature of SVM failing to capture non-linear patterns in the combined space.
• Success of Model Fusion: The proposed model fusion methods successfully recovered performance.
  • A composition approach using SVM (on SELDI) and RF (on Luminex) outputs fed into a Naïve Bayes classifier achieved an error rate of 8.82% and an AUC of 0.98.
  • This result was statistically significantly better (p < 0.0086) than the best data fusion approach (RF on merged data, AUC 0.88).
  • However, the model fusion results were only marginally better than using the Luminex data alone (AUC 0.98 vs. 0.98), suggesting that the Luminex panel contained the majority of the discriminative information in this specific study.

Significance and Claims
The paper claims that simple data merging is not a universal solution for combining heterogeneous bioinformatics data sources and can actively degrade predictive performance. The primary contribution is the demonstration that model fusion—combining the decision boundaries or soft outputs of specialized models rather than the raw data—can effectively leverage complementary information from disparate sources.

The authors modestly conclude that while their specific results on this pancreatic cancer dataset are difficult to generalize directly to all contexts, the existence of the problem (performance degradation via naive merging) indicates that maximizing the utility of multiple data sources requires moving beyond simple data fusion toward more sophisticated model-level integration strategies.