SpectraLLM: Uncovering the Ability of LLMs for Molecule Structure Elucidation from Multi-Spectral

1. Problem Statement

Molecular structure elucidation—the process of determining a molecule's chemical structure from spectroscopic data—is a fundamental challenge in chemistry, biology, and materials science. While techniques like Infrared (IR), Raman, UV-Vis, Nuclear Magnetic Resonance (NMR), and Mass Spectrometry (MS) provide complementary insights, existing automated approaches suffer from significant limitations:

• Modality Silos: Most state-of-the-art models are restricted to a single spectral modality (e.g., only MS or only NMR), failing to leverage the complementary nature of different spectroscopic techniques.
• Architectural Rigidity: Traditional methods often treat spectra as fixed numerical features or images, making it difficult to integrate heterogeneous data types (continuous vs. discrete) or contextual metadata (e.g., collision energy, solvent).
• Dependency on Databases: Many approaches rely on pre-compiled spectral libraries, limiting their ability to perform de novo structure prediction for novel compounds.

The authors propose that Large Language Models (LLMs), with their ability to perform symbolic reasoning and handle diverse inputs within a shared semantic space, offer a superior paradigm for unifying these heterogeneous spectroscopic signals.

2. Methodology: SpectraLLM

SpectraLLM is a unified, language-based framework designed to perform end-to-end molecular structure prediction from one or multiple spectral modalities.

A. Semantic-Driven Spectroscopic Reasoning

Instead of using convolutional neural networks (CNNs) or vision encoders to process raw spectra, SpectraLLM transforms spectral data into natural language text.

• Prompt Construction: Raw spectral peaks (wavenumbers, chemical shifts, m/z ratios) and intensities are extracted and formatted into structured textual descriptions.
• Contextualization: Experimental metadata (e.g., ionization mode, collision energy, instrument type) are included in the prompt to help the model contextualize the data.
• Unified Input: Multiple spectra (e.g., IR + NMR + MS) are concatenated into a single prompt, allowing the model to reason over complementary evidence simultaneously.

B. Model Architecture

• Backbone: The model utilizes Qwen3 (a 32B parameter LLM) as its foundation.
• Training Strategy: The backbone weights are frozen. The model is adapted using Low-Rank Adaptation (LoRA), which introduces lightweight trainable matrices. This preserves the model's general linguistic and reasoning capabilities while efficiently aligning it with spectroscopic tasks.
• Objective: The task is framed as an autoregressive sequence-to-sequence generation problem. The model predicts the SMILES string (a linear notation for chemical structures) token-by-token, conditioned on the spectral prompt.
• Loss Function: Standard cross-entropy loss is used to minimize the negative log-likelihood of the ground-truth SMILES sequences.

C. Data Curation

The model was trained on a massive, diverse corpus comprising over 5.5 million spectra paired with 940,000 unique molecules. Sources include:

• QM9s: Simulated IR, Raman, and UV-Vis spectra.
• Multimodal Spectroscopic Dataset: Simulated IR, MS, and multi-NMR (1H, 13C, HSQC).
• MassSpecGym & MassBank: High-resolution experimental MS and MS/MS data.
• Preprocessing: Spectra were processed to extract discrete peak features (positions and intensities), normalized, and serialized into JSON-like text prompts.

3. Key Contributions

1. Unified Language-Based Paradigm: SpectraLLM is the first model to represent both continuous (IR, Raman, UV-Vis, NMR) and discrete (MS) modalities in a shared language space, enabling joint reasoning without modality-specific encoders.
2. Multi-Spectral Synergy: The model supports flexible input configurations, allowing it to infer structures from single spectra or fuse multiple spectra. It demonstrates that performance improves monotonically as spectral diversity increases.
3. State-of-the-Art Performance: The model achieves superior results across four public benchmark datasets, outperforming specialized baselines in both single-modality and multi-modality settings.
4. Robustness and Generalization: SpectraLLM shows strong robustness to noise and generalizes effectively to real-world experimental data (e.g., MassBank, NIST) even when trained primarily on simulated data.

4. Experimental Results

The authors evaluated SpectraLLM on four datasets: QM9s, Multimodal Spectroscopic, MassSpecGym, and MassBank. Metrics included Validity, Tanimoto similarity (ECFP4 and MACCS), Cosine similarity, Maximum Common Edge Substructure (MCES), and Functional Group recovery.

• Unimodal Performance:

  • NMR: SpectraLLM achieved a Tanimoto similarity of 0.4151 (vs. 0.0433 for the baseline NMR2Struct) and reduced MCES error from 30.6 to 8.3.
  • Mass Spec: On MassBank, SpectraLLM achieved 98.44% validity and a Tanimoto of 0.1286, significantly outperforming DiffMS (0.0742) and Spec2Mol (0.0857).
  • Vibrational (IR/Raman): SpectraLLM doubled the performance of previous neural baselines on Raman data (Tanimoto 0.2500 vs. 0.1089).

• Multi-Spectral Fusion:

  • Combining modalities yielded substantial gains. For example, on the Multimodal Spectroscopic dataset, fusing NMR + IR + MS increased Tanimoto similarity to 0.4875 and Functional Group recovery to 0.8103, compared to lower scores for single modalities.
  • Case studies (Figure 2) demonstrated that IR is critical for identifying carbonyl groups, while Raman is essential for resolving polarizability-dependent substructures (e.g., imidazole rings), proving the models' ability to leverage complementary constraints.

• Ablation Studies:

  • Architecture: The unified LLM approach significantly outperformed a Vision-Language Model (VLM) pipeline, which suffered from cross-modal encoding artifacts and lossy intermediate representations.
  • Noise Robustness: The model degraded gracefully under simulated noise; performance only collapsed when multiple modalities were simultaneously corrupted.
  • Zero-Shot: Pretrained LLMs without fine-tuning failed completely (Tanimoto ~0.03), confirming that the performance stems from learning the spectrum-structure mapping, not memorizing pretraining data.

5. Significance and Conclusion

SpectraLLM represents a paradigm shift in automated chemical analysis. By reframing spectroscopic inference as a language task, it overcomes the architectural rigidity of traditional deep learning models.

• Scientific Impact: It enables de novo structure elucidation without reliance on spectral libraries, mimicking the way human experts integrate multiple data sources to resolve structural ambiguities.
• Scalability: The language-based approach is inherently scalable; adding new modalities or experimental conditions simply requires updating the text prompt format rather than redesigning the model architecture.
• Future Directions: The authors suggest this framework can be extended to other physical measurement domains, establishing a generalizable "scientific inverse solver" based on large language models.

The code and data are open-sourced, facilitating further research in language-based scientific reasoning.