MOLAR: Learning Multimodal Molecular Representations from Noisy Labels

Technical Summary: MOLAR – Learning Multimodal Molecular Representations from Noisy Labels

Problem Statement
Molecular property prediction often relies on labels derived from high-throughput assays, curated databases, or weak annotation pipelines, which are frequently noisy rather than reflecting clean biological ground truth. In multimodal molecular representation learning, where models integrate graph-based structural data (atoms, bonds, topology) with text-based semantic data (SMILES, descriptions, descriptors), treating these noisy recorded labels as reliable supervision can lead to models memorizing corrupted observations. This issue is exacerbated in multimodal settings: standard fusion or alignment strategies may propagate label-induced errors across modalities, and graph–text disagreements are often misinterpreted as alignment failures rather than potential indicators of label corruption, insufficient evidence in one modality, or genuine uncertainty. Existing noisy-label learning methods primarily address unimodal classification and do not explicitly distinguish latent molecular properties from recorded noisy labels, nor do they provide a principled framework for handling modality-specific evidence under unreliable supervision.

Methodology: The MOLAR Framework
The authors propose MOLAR, a noise-aware framework designed to separate latent clean-property inference from recorded-label observation. The framework operates through four key modules:

1. Molecular Evidence Initialization:

   • Graph View: A molecular graph $G_i$ is encoded into a representation $z^g_i$ using a graph encoder (e.g., GIN).
   • Text View: A text-derived view $T_i$ (e.g., SMILES, descriptions) is encoded into $z^t_i$ using a text encoder (e.g., BiomedBERT).

2. Residual Natural-Parameter Evidence Composition:
   Instead of concatenating features or forcing alignment, MOLAR treats graph and text views as sources of residual evidence for a latent clean categorical distribution.

   • Learnable functions map representations to class-wise residual natural parameters: $u^g_i = C(\phi_g(z^g_i))$ and $u^t_i = C(\phi_t(z^t_i))$, where $C$ is a centering operator removing common additive offsets.
   • A base vector $b$ represents class-level priors.
   • The clean logits are formed by additive composition: $\ell_i = C(b + u^g_i + u^t_i)$.
   • The clean posterior $p_i$ is derived via softmax normalization. This formulation allows the model to quantify the relative contribution of each modality ($M^g_i, M^t_i$) and interpret modality-specific evidence.

3. Categorical Label-Observation Channel:
   To utilize noisy recorded labels $\tilde{y}_i$ without treating them as ground truth, MOLAR introduces a probabilistic channel $O$ that maps the latent clean class $y_i$ to the recorded label $\tilde{y}_i$.

   • $O_{ac} = p(\tilde{y}_i = a | y_i = c)$.
   • The channel is parameterized to be diagonal-dominant (allowing for label transitions but prioritizing correct mapping) and is learned during training.
   • The recorded-label distribution $\tilde{p}_i$ is computed as $O p_i$. The model is trained to maximize the likelihood of the recorded label under this distribution, rather than directly fitting $p_i$ to $\tilde{y}_i$.
   • This channel naturally yields a posterior reliability score $\rho_i$, representing the probability that the recorded label matches the latent clean class.

4. Noise-Aware Learning Objective:
   The total loss function combines three terms:

   • Recorded-Label Likelihood ($L_{sup}$): Negative log-likelihood of the recorded label based on the marginalized distribution $\tilde{p}_i$.
   • Evidence-Conflict Regularization ($L_{conf}$): Penalizes contradictory evidence between graph and text views, weighted by the posterior reliability $\rho_i$. High-confidence contradictions are penalized more heavily when the label is likely reliable.
   • Perturbation-Consistency Regularization ($L_{pert}$): Encourages the clean posterior to remain stable under label-preserving molecular perturbations (e.g., feature masking), ensuring robustness against noise.

Key Contributions

1. Problem Formulation: The paper identifies and formalizes three core challenges in noisy-label multimodal molecular learning: (a) separating latent properties from noisy labels, (b) utilizing noisy labels as supervision via a label-observation channel, and (c) interpreting graph–text disagreement under unreliable supervision.
2. Framework Design: MOLAR introduces a probabilistic framework that composes multimodal views as residual evidence for a clean posterior and links this to noisy labels through a learnable observation channel, avoiding direct fitting to corrupted targets.
3. Interpretability: The framework provides diagnostic tools, including posterior reliability scores ($\rho_i$) and modality-specific evidence contributions, allowing for the interpretation of label trustworthiness and modality utility.
4. Empirical Validation: Extensive experiments demonstrate state-of-the-art performance on both naturally noisy benchmarks (MF-PCBA-Noisy7) and controlled label-flipping benchmarks (MoleculeNet).

Experimental Results

• Datasets: Evaluation was conducted on MF-PCBA-Noisy7 (seven tasks with naturally noisy single-dose screening labels and cleaner dose-response confirmatory labels) and four MoleculeNet benchmarks (HIV, BACE, BBBP, ClinTox) with controlled symmetric label flipping (up to 70%).
• Performance: MOLAR consistently outperformed representative baselines across four categories: supervised graph neural networks (GCN, GAT, GIN), molecular pretraining methods (GraphCL, GROVE, Uni-Mol), multimodal fusion methods (Tri-SGD, MMSG, ProtoMol), and robust/noisy-label learning methods (OMG, SPORT, RTGNN).
  • On MF-PCBA-Noisy7, MOLAR achieved the highest average ROC-AUC (71.30%), showing significant gains on tasks with severe label inconsistency.
  • On MoleculeNet with 30% label flipping, MOLAR maintained superior performance (78.45% average ROC-AUC) compared to all baselines.
  • Robustness analysis showed that while baseline performance degraded rapidly as noise levels increased, MOLAR maintained stable accuracy, particularly on BBBP and ClinTox.
• Ablation Studies: Removing the conflict regularization, perturbation consistency, or either modality (graph/text) resulted in performance degradation, confirming the necessity of the joint design. Replacing the learned channel with a shared one also reduced performance.
• Diagnostics:
  • Reliability: The posterior reliability $\rho_i$ showed a strong Spearman correlation (0.98) with empirical SD–DR agreement rates, outperforming entropy and conflict scores in detecting label inconsistencies.
  • Visualization: Case studies on flipped samples demonstrated that MOLAR could correctly predict the clean label despite corrupted recorded labels, attributing decisions to specific molecular substructures rather than memorizing noise.

Significance
The paper claims that MOLAR provides a principled approach to multimodal molecular representation learning in the presence of noisy supervision. By explicitly decoupling clean property inference from label observation and modeling the relationship between modalities and labels probabilistically, the framework mitigates the risk of memorizing corrupted data. The ability to derive posterior reliability and modality-specific evidence offers a diagnostic capability that is crucial for interpreting model behavior in real-world drug discovery scenarios where labels are inherently uncertain. The results suggest that separating latent properties from noisy observations is a critical step for improving generalization and robustness in molecular property prediction.