Influence of molecular representation and charge on protein-ligand structural predictions by popular co-folding methods

This study reveals that the choice of molecular input format (CCD vs. SMILES) significantly impacts protein-ligand structural predictions by popular deep learning tools like AlphaFold 3 and Boltz-2, often more so than protonation state, highlighting the urgent need for format-independent consistency and improved handling of protonation in future algorithmic developments.

The Gist

Imagine you have a super-smart robot architect named AlphaFold 3, along with three of its equally talented friends: Boltz-2, Chai-1, and Protenix-v1. These robots are trained to build 3D models of how proteins (the body's tiny machines) grab onto specific chemical keys (ligands) to unlock them. This is crucial for designing new medicines.

The scientists behind this paper decided to put these robots through a "stress test" to see if they truly understand the chemistry they are modeling. They asked two simple questions:

1. Does the robot care if the key is charged? (Like a magnet with a positive or negative pole).
2. Does it matter how you describe the key to the robot? (Do you give it a photo, a written description, or a code?)

Here is what they found, explained through some everyday analogies.

1. The "Identity Crisis" of the Robots

The researchers gave the robots two very simple chemicals to work with: methylamine (a tiny, neutral molecule) and methylammonium (the same molecule but with a positive charge, like a tiny magnet).

• The Expectation: If you give a robot a magnet, it should stick to the metal part of the machine. If you give it a neutral rock, it shouldn't stick. The robots should know the difference between a magnet and a rock.
• The Reality: The robots were confused.
  • The Charge Problem: When the scientists told the robot, "Hey, this molecule is positively charged," the robot often ignored it. It built the model exactly the same way as if the molecule were neutral. It was like telling a dog, "This is a fire hydrant," and the dog still treating it like a tree.
  • The Format Problem: Here is the weird part. The robots cared way more about how the molecule was described than what the molecule actually was.
    • If you described the molecule using a standard chemical code (called CCD), the robot built one shape.
    • If you described the exact same molecule using a different text code (called SMILES), the robot built a completely different shape.
    • Analogy: Imagine you ask a chef to bake a cake. If you write the recipe in "Metric units," they make a chocolate cake. If you write the exact same recipe in "Imperial units," they make a carrot cake. The ingredients didn't change, but the way you wrote them down changed the result.

2. The "Shrinking" Molecules

The robots also struggled to get the size of the molecules right.

• The Expectation: Chemical bonds (the glue holding atoms together) have a specific, unchangeable length, like the distance between two rungs on a ladder.
• The Reality: The robots kept making the molecules too small. In some cases, they squished the atoms so close together that they were practically touching, which is physically impossible.
• Analogy: It's like a 3D printer that keeps printing a toy car, but the wheels are always half the size they should be, and sometimes the wheels are so small they look like dots.

3. The "Lost Keys"

The researchers tested the robots on two specific protein targets:

• The Dopamine Receptor (DRD1): This is like a lock in the brain that usually grabs onto charged keys. The robots mostly got this right, placing the charged key near the lock.
• The BarA Sensor: This is a different protein that grabs onto a specific type of acid.
• The Reality: For the BarA sensor, none of the robots found the right spot. They all put the key in the wrong place, like trying to fit a key into a door that isn't even there. This suggests the robots are just "memorizing" patterns from their training data rather than truly understanding how chemistry works.

The Big Takeaway

The main conclusion of the paper is a warning label for scientists and doctors: Don't trust these robots blindly yet.

• The "Input Format" Bug: If you get a result using one type of file format, and a different result using another, the difference might just be a glitch in how the robot reads the file, not a real scientific discovery.
• The "Charge" Blindness: The robots aren't good at understanding that a molecule's electrical charge changes how it behaves. They need to be retrained to understand basic physics.

In short: These AI tools are incredibly powerful, like a Ferrari engine. But right now, the steering wheel is a bit loose, and the GPS sometimes gets confused if you type the address in a different font. Until the developers fix these bugs, we need to be very careful about how we use these predictions for real-world medicine.

Technical Summary

1. Problem Statement

Recent deep learning-based "co-folding" tools (e.g., AlphaFold 3, Boltz-2, Chai-1, Protenix-v1) have revolutionized the prediction of protein-ligand complex structures. However, their reliability regarding ligand charge states and input molecular representations remains unverified.

• The Core Issue: Ligands in biological systems often exist in different protonation states (e.g., neutral vs. charged) depending on pH, which drastically alters their binding behavior. Furthermore, these tools accept ligands via different input formats: SMILES (string notation) and CCD (Chemical Component Dictionary codes).
• The Hypothesis: It is unclear whether these algorithms correctly utilize the explicitly specified charge of a ligand to predict binding poses and whether the choice of input format (SMILES vs. CCD) introduces systematic biases or inconsistencies.
• Motivation: If algorithms fail to distinguish between charged and neutral forms of simple titratable molecules (like amino and carboxy groups), their utility in drug discovery and understanding pH-dependent binding is compromised.

2. Methodology

The authors conducted a systematic benchmark of four popular co-folding tools: AlphaFold 3, Boltz-2, Chai-1, and Protenix-v1.

• Test Cases:

  • Ligands: Two simple titratable molecules ubiquitous in proteins:
    1. Methylamine/Methylammonium: Tested in the context of the human Dopamine D1 Receptor (DRD1). The charged form is expected to form an ionic bridge with residue D103, while the neutral form should not.
    2. Acetic Acid/Acetate: Tested in the context of the E. coli BarA kinase sensor domain. The protonated form is the physiological ligand expected to bind a specific hydrophobic/polar pocket.
  • Input Formats:
    • CCD: Used codes NME (methylamine), 3P8 (methylammonium), ACY (acetic acid), ACT (acetate).
    • SMILES: Used strings CN, C[NH3+], CC(=O)O, CC(=O)[O-].
    • Note: Chai-1 only accepts SMILES; CCD was not tested for this tool.

• Experimental Setup:

  • Hardware: NVIDIA GeForce RTX 4090 GPUs.
  • MSA Generation: Performed locally (AlphaFold 3) or via a local Colabfold server (others). MSAs were generated for the apo-protein first, then reused for complex prediction.
  • Sampling:
    • AlphaFold 3: 100 seeds × 5 diffusion samples.
    • Boltz-2: 500 samples (BarA) or 5 runs × 100 samples (DRD1).
    • Chai-1 & Protenix-v1: 50 seeds × 10 samples.
  • Analysis:
    • Geometric Validation: Bond lengths (C-N, C-C, C-O) compared against quantum chemical reference ranges.
    • Positional Analysis: Ligand coordinates were superimposed on the protein backbone. Principal Component Analysis (PCA) was used to visualize ligand distribution.
    • Statistical Comparison: The Kolmogorov–Smirnov (K-S) statistic was calculated to quantify the difference between distributions (e.g., charged vs. neutral, CCD vs. SMILES).

3. Key Results

A. Bond Length Accuracy

• Systematic Underestimation: All tools predicted bond lengths significantly shorter than quantum chemical reference values.
  • Example: Methylamine C-N bonds were predicted around 1.26–1.32 Å (Reference: 1.45–1.48 Å).
• Extreme Outliers: Protenix-v1 generated biologically impossible bond lengths (as short as 0.075 Å) for methylammonium.
• Charge Insensitivity: Most tools (Chai-1, Protenix-v1) showed no sensitivity to the specified charge; bond lengths remained identical for neutral and charged forms.
• Format Dependence: Bond length distributions varied significantly between SMILES and CCD inputs for the same tool and molecule.

B. Ligand Binding Poses

• DRD1 (Methylamine/Methylammonium):
  • Charge Effect: Contrary to expectations, the algorithms generally failed to distinguish between the charged (methylammonium) and neutral (methylamine) forms. Both were often placed in similar positions, ignoring the expected ionic bridge with D103.
  • Format Effect: The input format (CCD vs. SMILES) had a stronger impact on the predicted position than the charge itself. For example, Boltz-2 showed extreme sensitivity to the input format.
• BarA (Acetic Acid/Acetate):
  • None of the algorithms successfully recapitulated the known putative binding site (L95, T98, I136).
  • Only AlphaFold 3 showed a slight difference in predictions between acetate and acetic acid when using CCD, but the overall binding configuration was incorrect.

C. Statistical Findings

• The K-S statistic revealed that distributions of ligand positions often differed more between input formats (CCD vs. SMILES) than between charge states (neutral vs. charged).
• Tools like Chai-1 and Protenix-v1 produced highly diverse and often incorrect binding poses, while AlphaFold 3 and Boltz-2 were more consistent but still failed to model charge-dependent binding correctly.

4. Key Contributions

1. Identification of Format Bias: The study provides the first systematic evidence that popular co-folding tools produce different structural predictions based solely on the input file format (SMILES vs. CCD), even when the chemical identity is identical.
2. Charge Insensitivity: It demonstrates that current state-of-the-art models largely ignore explicitly specified ligand charges, failing to apply basic physico-chemical principles (e.g., electrostatic attraction/repulsion) to determine binding poses.
3. Geometric Deficiencies: The paper highlights severe inaccuracies in predicted bond lengths, with some tools generating non-physical geometries (e.g., 0.075 Å bonds).
4. Benchmarking Framework: Establishes a rigorous protocol for testing co-folding tools using simple titratable probes to assess charge and format sensitivity.

5. Significance and Implications

• Reliability Concerns: The findings suggest that current co-folding results must be interpreted with extreme caution, particularly in drug discovery scenarios where protonation states dictate binding affinity.
• Training Data Limitations: The failure to distinguish charges likely stems from training data (PDB) where protonation states are often ambiguous or missing (X-ray/cryo-EM do not directly show protons).
• Pathways for Improvement: The authors propose two critical directions for future algorithm development:
  1. Format Invariance: Ensuring that SMILES and CCD inputs yield identical structural predictions.
  2. Protonation-Aware Training: Integrating explicit protonation steps and charge states into training pipelines, potentially augmented by physics-based refinement or QM/MM calculations.
• Practical Advice: Users should test all possible input formats and charge states when using these tools and treat charge-induced differences as artifacts rather than biological insights until the models are improved.

In conclusion, while deep learning co-folding tools have advanced structural biology, they currently lack the physical rigor to handle simple electrostatic and representation nuances, limiting their immediate applicability for precise ligand binding prediction.