Technical Summary: MolSight

Problem Statement

Modern Molecular Property Prediction (MPP) has largely moved away from the universally available 2D skeletal diagrams (bond-line structures) that chemists have used for centuries. Instead, the field relies on modality-specific approaches: Graph Neural Networks (GNNs) requiring explicit covalent topology construction, 3D transformer models demanding expensive conformer generation, and billion-parameter Large Language Models (LLMs) utilizing instruction tuning. While effective, these methods impose significant computational overheads and data-engineering bottlenecks, particularly for high-throughput screening of millions of compounds. The central hypothesis of this work is whether a general-purpose vision encoder, guided by appropriate pre-training, can learn to read 2D molecular diagrams directly to achieve competitive property prediction, thereby eliminating the need for graphs, 3D conformers, or text decoding at inference.

Methodology

The authors present MolSight, a systematic large-scale study evaluating vision-based MPP. The methodology involves three primary axes:

1. Visual Architectures: The study evaluates 10 distinct architectures, categorized into:

   • Pure-vision encoders: ResNet-18, ResNet-50, DINOv2, and DINOv3.
   • Vision-language (VL) encoders: SigLIP, SigLIP2, OpenCLIP, Long-CLIP, MetaCLIP, and EVA-CLIP.
   • All models process standardized 224×224 RGB bond-line diagrams rendered via RDKit.

2. Pre-training Strategies: Seven strategies are tested to determine how best to encode chemical knowledge into the vision backbone:

   • S1 (Baseline): Supervised fine-tuning only.
   • S2 (Image-Image): Self-supervised contrastive learning (SimCLR) on augmented image pairs.
   • S3 (Image-SMILES): Cross-modal alignment using SigLIP loss between images and canonical SMILES strings.
   • S4 (Image-Description): Alignment with natural language molecular descriptions.
   • S5 (GNN Distillation): A hybrid loss function that distills representations from a frozen Uni-Mol (3D GNN) teacher into the vision encoder, combined with property regression.
   • S6 (Curriculum): Strategy S5 augmented with a chemistry-informed curriculum scheduler.
   • S7 (Joint SLIP): Simultaneous training of Image-Image and Image-SMILES objectives.

3. Chemistry-Informed Curriculum: A key methodological contribution is the partitioning of the pre-training corpus (2 million molecules) into five tiers based on structural complexity. Five descriptors—Scaffold Decoration, Functional-Group Rarity, Conjugation Extent, Aromatic Substitution Complexity, and Bertz Topological Complexity—are used to assign molecules to tiers (T0: simple hydrocarbons to T4: complex macrocycles). Training proceeds progressively, exposing the model to simpler structures before introducing complex ones.

4. Downstream Evaluation: Performance is evaluated on 10 tasks across three categories:

   • Physical-property regression: ESOL (solubility) and LD50 (toxicity).
   • Drug-discovery classification: BBBP, BACE, ClinTox, Tox21, and HIV (from MoleculeNet).
   • Quantum-chemistry prediction: QM9 (atomization energy and HOMO-LUMO gap) and PCQM4Mv2 (HOMO-LUMO gap).

Key Contributions

1. Comprehensive Pre-training Study: The first systematic evaluation of 10 architectures under 7 pre-training strategies across 10 downstream tasks, establishing an empirical map of which combinations best encode chemical knowledge.
2. Chemistry-Informed Curriculum: The proposal of a structural complexity-based curriculum that consistently outperforms random sampling across all tested architectures, demonstrating that sequencing training examples from simple to complex improves representation learning.
3. Competitive Image-Only Inference: Demonstration that a single rendered 2D image processed by a vision encoder is sufficient for competitive MPP. The best configuration achieves top results on 5 of 10 benchmarks and top-two on all 10, using only a 2D image at inference.

Results

• Strategy Efficacy: The pre-training strategy significantly outweighs architecture choice. Curriculum-augmented GNN distillation (S6) consistently ranks first across all tasks.
• Performance Gains:
  • Quantum Tasks: S6 reduces PCQM MAE by 82.7% (from 0.549 to 0.095) compared to Image-SMILES (S3) and by 19.1% compared to GNN distillation without curriculum (S5). QM9-∆ε MAE is reduced by 31.7%.
  • Drug Discovery: MolSight (S6) achieves 93.7% ROC-AUC on BBBP and 90.7% on Tox21, outperforming several graph-based pre-training baselines.
  • Physical Properties: The model achieves an ESOL RMSE of 0.627 and LD50 MAE of 0.369.
• Comparison to State-of-the-Art:
  • MolSight outperforms Uni-Mol (a leading 3D-GNN) on 8 of 10 benchmarks and MolVision (a VLM) on 7 of 10.
  • It significantly outperforms LLMs (GPT-4o, Janus-Pro) on all 10 tasks, indicating that in-context learning cannot substitute for structured chemical pre-training.
  • Efficiency: The ViT-B/16 model requires only 17.6 GFLOPs per inference, which is 80× lower than MolVision and >500× lower than GPT-4o.
• Embedding Geometry: t-SNE analysis reveals that curriculum training (S6) organizes embeddings into smooth, continuously graded manifolds aligned with quantum properties (e.g., atomization energy), whereas non-curriculum baselines show coarse clustering.

Significance and Claims

The paper claims that chemical insight can be derived from sight alone. It challenges the necessity of complex multi-modal inputs (graphs, 3D conformers, billion-parameter LLMs) for high-performance MPP. The authors argue that while 3D information is valuable for specific quantum tasks, a vision encoder pre-trained with the right strategy (specifically curriculum-augmented GNN distillation) can capture sufficient structural and electronic cues from 2D diagrams to rival specialized models.

The study highlights that pre-training strategy matters far more than architecture choice. The chemistry-informed curriculum provides consistent, computationally negligible improvements, suggesting that the ordering of training data by structural complexity is a critical factor in learning robust molecular representations.

Limitations acknowledged by the authors:

• The model remains slightly below 3D-GNN methods on tasks with strong 3D dependence (e.g., PCQM4Mv2).
• Very large molecules (>80 heavy atoms) can saturate the 224×224 rendering resolution.
• The approach relies on a single rendering library (RDKit), showing sensitivity to depiction styles (e.g., different bond widths or libraries like PIKAchU).

The work concludes that image-only inference is a surprisingly effective and highly scalable alternative for molecular property prediction, offering a pathway to million-compound screening with significantly reduced computational costs.