SciCore-Mol: Augmenting Large Language Models with Pluggable Molecular Cognition Modules

Technical Summary: SciCore-Mol

Problem Statement
Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities but face a fundamental mismatch when applied to scientific domains like chemistry. Scientific data, such as molecular structures and reaction conditions, is inherently topological, geometric, and continuous. In contrast, LLMs are designed for discrete symbolic sequences. Current approaches attempt to bridge this gap by linearizing molecular graphs into text strings (e.g., SMILES or SELFIES). However, this linearization fails to explicitly expose the topological and geometric invariances central to chemical reasoning, leading to information loss, semantic noise, and unreliable predictions on structure-sensitive tasks. Furthermore, existing tool-augmented systems often treat scientific modules as external black boxes, requiring intermediate data to be compressed back into text before being processed by the LLM. This "text-level interface" creates reasoning bottlenecks and prevents the LLM from accessing dense, continuous molecular information.

Methodology: SciCore-Mol Framework
The authors propose SciCore-Mol, a modular framework that augments an LLM backbone with three deeply integrated, pluggable cognitive modules. Unlike previous approaches that rely on text-only feedback, SciCore-Mol couples these modules to the LLM through learned representation interfaces at the hidden-state level.

1. Topological Perception Module: Built on a Geometric Vector Perceptron (GVP) network, this module encodes 3D molecular geometry (generated via RDKit) into SE(3)-equivariant representations. It integrates scalar atom-level properties and 3D coordinate vectors via message passing. The resulting geometric embedding is projected into the LLM's hidden space and injected as a "Virtual Structural Token," allowing the LLM to attend to 3D structural information alongside text tokens.
2. Molecular Generation Module: To address the low validity of autoregressive SMILES generation, this module employs a Latent Diffusion Transformer (DiT). It operates in a continuous molecular latent space, conditioned on the LLM's hidden states. A pre-trained SMILES autoencoder maps molecules to latent vectors, and the diffusion process iteratively denoises these vectors to generate structurally coherent molecules, which are then decoded back to SMILES.
3. Reaction Sensing Module: Built on a numerical-sensitive Transformer, this module handles inter-molecular reasoning. It processes reaction data hierarchically using multi-channel composite encodings that combine GVP-derived geometric embeddings, stoichiometric quantities (moles, mass), and functional roles (reactant, catalyst, etc.). This allows for joint prediction of reaction products, yields, and missing reactants.

Training Strategy
The system utilizes a progressive training strategy to align heterogeneous modules with the LLM backbone:

• Stage 1 (Component Pre-training): Modules are pre-trained independently. The LLM backbone undergoes continual pre-training on a chemistry-enriched corpus (~400M tokens) with KL-divergence regularization against a frozen base model to prevent catastrophic forgetting.
• Stage 2 (Cross-Modal Alignment): The LLM backbone is frozen while the GVP encoder and adapters are trained to align geometric representations with natural language descriptions using a contrastive objective.
• Stage 3 (Joint Multi-Task Training): The full system is jointly optimized on a 300K instruction-response corpus covering diverse tasks (synthesis, property prediction, generation, etc.).
• Stage 4 (Task-Specific Fine-tuning): Optional fine-tuning allows for specialization in specific downstream tasks by selectively freezing or unfreezing modules.

Key Contributions

• Architectural Innovation: A unified system that integrates topology-aware perception, latent diffusion generation, and reaction-aware reasoning directly into the LLM's hidden state, eliminating the information bottleneck of text-only tool interfaces.
• Modularity: The "pluggable" design allows modules to be independently enabled, disabled, or upgraded (e.g., swapping the GVP encoder for a stronger 3D architecture) without retraining the entire system.
• Performance: The authors demonstrate that an 8B-parameter SciCore-Mol system achieves competitive, and in some dimensions superior, performance compared to much larger proprietary models (e.g., GPT-4o, GPT-5) and other open-source chemistry models.

Experimental Results
Evaluated across five capability dimensions (Knowledge Core, Mol-Text Translation, Molecule Generation, Quantitative Prediction, and Synthesis Reasoning) using benchmarks like SMolInstruct, MMLU-Chemistry, ChemBench4K, ORD, and MoleculeNet:

• Balanced Performance: SciCore-Mol exhibits the most balanced capability profile among models of similar scale, outperforming baselines in molecule generation, quantitative prediction, and synthesis reasoning while maintaining strong general knowledge.
• Structure-Sensitive Tasks: Ablation studies confirm that the GVP encoder significantly improves performance on tasks requiring 3D information (e.g., Tox21, HIV classification, QM9 regression), where text-only models struggle.
• Reaction Reasoning: Removing the Reaction Sensing Module leads to severe degradation in product and yield prediction validity, confirming the necessity of structured modeling for reaction-level reasoning.
• Generalization: The model shows strong out-of-distribution generalization on the DrugR benchmark (released after pre-training), suggesting it learns transferable structural priors rather than relying solely on memorization.

Significance and Claims
The paper claims that SciCore-Mol provides a systematic blueprint for equipping LLMs with scientific expertise through decoupled, pluggable, and flexibly orchestrated modules. The authors argue that deep, embedding-level integration of specialized modules is more effective than relying solely on text-based molecular representations or loose tool-augmented reasoning. This framework is presented as a general template for scientific foundation models that require heterogeneous data integration, with direct implications for drug design, chemical synthesis, and broader scientific discovery. The work emphasizes that while the current system is designed for small molecules, the modular architecture offers a path toward supporting larger entities like proteins and polymers in future iterations.