🧬 biology

Carried but Conditionally Used: A Frozen Held-Out Audit of Whether Chirality-Aware Molecular Fingerprints Predict Measured Same-Assay Enantioselectivity

This study employs a prospectively specified, cryptographically frozen audit to demonstrate that while radius-3 Morgan fingerprints effectively distinguish enantiomers, their ability to predict measured same-assay enantioselectivity under scaffold extrapolation is real but strongly target-dependent, yielding significant results for only a subset of targets while others show null or negative predictive skill.

Original authors: Dylan Ashraf

Published 2026-09-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dylan Ashraf

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the world of medicine, the shape of a molecule matters as much as its ingredients. Many drugs are built from atoms that can be arranged in two ways that are mirror images of each other, much like a left hand and a right hand. These mirror-image versions are called enantiomers. While they look identical on paper, they often behave very differently inside the human body. One version might cure a disease, while its mirror image does nothing or even causes harm. Because of this, scientists need computer tools that can tell these mirror images apart and predict which one will work better. For years, researchers have built digital maps of molecules to help with this, but a lingering question remained: do these maps actually capture the difference between the mirror images, and can they use that difference to predict real-world results?

A new study by Dylan Ashraf, a researcher at American High School, tackles this question with a rigorous, step-by-step audit. Instead of simply building a new model and hoping for the best, the researcher set up a strict test to separate two distinct ideas. The first idea is "carrying" the information: does the digital map actually record the difference between the mirror images? The second idea is "using" that information: can a computer model look at that map and successfully predict which mirror image is more powerful in a lab experiment? To answer this, the researcher used a massive, public database of chemical data containing thousands of pairs of mirror-image molecules. The study focused on pairs where both versions had been tested in the exact same lab experiment, ensuring a fair comparison.

The investigation began by checking if the digital maps, known as fingerprints, could even distinguish between the mirror images. The researcher found that when the map looked at a molecule with a specific level of detail, it successfully recorded the difference between the left-handed and right-handed versions almost every time. This meant the information was being "carried" correctly. However, having the information is not the same as being able to use it to make a prediction. The next step was to see if a computer model could take these detailed maps and accurately guess the difference in strength between the two mirror images for various biological targets, such as proteins involved in disease.

The results were mixed, revealing that the ability to predict depends heavily on the specific target being studied. Out of eleven different biological targets that were tested without prior knowledge of the outcome, the computer model successfully predicted the difference in strength for five of them. For these five targets, the model could look at the mirror images and correctly identify which one would be more potent. However, for the other six targets, the model failed to find a pattern, performing no better than random guessing or even worse. This finding is crucial because it shows that just because a computer can see the difference between mirror images, it does not mean it can always predict how that difference will play out in a living system. The success was not universal; it was conditional on the specific biology of the target.

The study also examined whether the complexity of the computer model mattered. The researcher tested two different types of learning methods: one that looks for simple, straight-line relationships and another that looks for complex, twisting patterns. Surprisingly, both methods performed about the same. This suggests that the part of the signal that helps predict the difference between mirror images is likely straightforward and additive, rather than hidden in complex, non-linear interactions. Furthermore, the researcher checked if the model was simply making things up. By testing on targets where no real difference existed and on controls where the mirror images were chemically identical, the study confirmed that the model did not automatically create false predictions. When there was no signal to find, the model correctly returned no result.

One of the most interesting parts of the study involved checking if the size of the difference in strength between the mirror images made prediction easier. Intuitively, one might think that if the difference is huge, it would be easier for a computer to spot. The data showed a hint of this relationship across the entire set of targets, but when the researcher looked strictly at the targets they had not seen before, this connection became weak and statistically uncertain. This suggests that while a larger difference might help, it is not the only factor, and other biological complexities likely play a major role. The study also clarified a technical detail about how these digital maps work. It proved that the occasional failure to distinguish between mirror images at a lower level of detail was due to the limited scope of the view, not a flaw in the computer's hashing system.

Ultimately, this work provides a clear, disciplined answer to a long-standing question in drug discovery. It confirms that modern digital maps can carry the information about molecular mirror images, but it also demonstrates that using that information to predict real-world results is not guaranteed. The ability to predict depends entirely on the specific biological target. For some targets, the signal is strong enough to be predicted; for others, it is not. This does not mean the technology is broken, but rather that it has limits. The study concludes that while we have the tools to see the difference between mirror images, successfully predicting which one will work requires a deeper understanding of the specific biological context, and that for some targets, the current methods may not be enough to make a reliable prediction.

Technical Summary: "Carried but Conditionally Used: A Frozen Held-Out Audit of Whether Chirality-Aware Molecular Fingerprints Predict Measured Same-Assay Enantioselectivity"

Problem Statement
Chirality is a first-order determinant of drug potency, with enantiomers often differing by orders of magnitude in target affinity. While many standard two-dimensional molecular representations (e.g., standard Morgan fingerprints without chirality flags) fail to distinguish enantiomers, "chirality-aware" representations exist. However, a critical gap remains in the literature: it is often unclear whether a representation that technically carries stereochemical information actually uses that information to predict experimentally measured biological differences out-of-sample. Specifically, does a model leveraging chirality-aware fingerprints successfully predict the potency differences ( $\Delta$ potency) between enantiomers under scaffold extrapolation, or does it merely encode the information without utilizing it for predictive gain?

Methodology
The study employs a prospectively specified, cryptographically frozen analysis protocol to audit 14 specific biological targets using data from ChEMBL 37.

Data Curation: The author isolated high-quality, same-assay enantiomer pairs (IC50, binding assays, confidence score 9) to ensure internal controls. Inclusion required $\ge$ 80 unique enantiomer pairs and $\ge$ 40 Bemis–Murcko scaffold groups per target to support scaffold-based cross-validation.
Representation: Primary features were radius-3 Morgan fingerprints (4,096 bits) with chirality enabled. Sensitivity checks included radius-2 and count fingerprints, while achiral radius-3 fingerprints served as collision-floor controls.
Modeling Task: The task was to predict the signed difference in potency ( $\Delta y = pChEMBL(A) - pChEMBL(B)$ ) for enantiomer pairs. To enforce physical symmetry, the author constructed an exactly antisymmetric model using a symmetrized estimator: $\Delta\hat{y}(\delta) = [h(\delta) - h(-\delta)]/2$ , where $\delta$ is the signed difference of the fingerprint vectors.
Evaluation Protocol:
- Scaffold Extrapolation: Primary evaluation used 5-fold scaffold-grouped cross-validation to test generalization to unseen chemical scaffolds.
- Held-Out Separation: Three targets (BACE1, BTK, JAK3) were designated as a development cohort for pipeline construction and null calibration. The remaining 11 targets formed a strictly held-out confirmatory cohort.
- Statistical Rigor: Significance was assessed using scaffold-cluster-level sign-flip nulls and pair-level label permutations. Benjamini–Hochberg (BH) false discovery rate (FDR) correction was applied only within the 11 held-out targets, excluding the development targets from the correction family to prevent data leakage.
- Metrics: The primary metric was a normalized zero- $\Delta$ skill score ( $S_\Delta$ ), defined as $1 - \text{MAE}(\hat{\Delta}, \Delta) / \text{MAE}(0, \Delta)$ . Secondary metrics included Spearman correlation and directional accuracy.

Key Results

Carry vs. Use:
- Carry (Representation): At radius 3, chirality-aware Morgan fingerprints almost completely distinguish enantiomers. The study found that elevated collision rates at radius 2 were due to neighborhood-radius limitations (insufficient context) rather than hashing artifacts, as collisions disappeared at radius 3 and remained zero across bit lengths (1,024–8,192) and unhashed identifiers.
- Use (Prediction): Predictive utility is real but highly target-dependent. In the 11 held-out targets, 5 targets (JAK1, JAK2, ERK2, ROR- $\gamma$ , HDAC6) showed statistically significant ( $q \le 0.0033$ ) positive skill in predicting enantioselectivity under scaffold extrapolation.
- Null/Negative Cases: Several targets, including the achiral control and specific biological targets like hERG (KCNH2) and JAK3, returned null or negative skill scores, demonstrating that the procedure does not artificially manufacture predictability.
Model Performance:
- Ridge regression and Random Forest models performed comparably (median difference in $S_\Delta$ of +0.005), suggesting that much of the accessible stereochemical signal in signed-fingerprint space is additive rather than requiring complex non-linear interactions.
Signal-to-Variability Association:
- A descriptive trend was observed where targets with higher median potency differences relative to measurement variability ( $R_t$ ) tended to show higher skill. However, this association did not reach statistical significance in the held-out cohort ( $p=0.20$ for $S_\Delta$ ). The author notes that HDAC6, despite a low median signal, showed high skill due to a heavy upper tail of large potency differences, indicating that median-based signal summaries may be too coarse.
Boundary Cases:
- EGFR showed positive skill only when the development targets were included in the correction family ( $q=0.0498$ ) but failed to reach significance in the strict held-out family ( $q=0.0503$ ), highlighting the sensitivity of the result to the correction family definition.

Significance and Claims
The paper's primary contribution is a claim-disciplined audit that reframes the definition of "chirality-aware" from representational capacity ("carry") to measured out-of-sample predictive utility ("use").

Separation of Concepts: The work explicitly separates the deterministic property of a fingerprint carrying stereochemistry from the statistical property of a model using that information to predict biology. It concludes that while carry is near-complete at radius 3, use is conditional and target-specific.
Reproducibility and Rigor: By employing a cryptographically frozen protocol, prospectively defined development/held-out splits, and strict FDR correction within the held-out family, the study provides a robust, non-exaggerated assessment of current capabilities.
Modest Conclusions: The author does not claim a universal law governing enantioselectivity prediction. Instead, they report a balanced landscape of positive, null, and negative findings. They suggest that the signal-to-variability hypothesis is worth testing at a larger scale with tail-aware statistics but remains unconfirmed in this study.
No New Model or Descriptor: The study does not introduce a new molecular representation or a new modelability index; rather, it provides a rigorous evaluation framework for existing tools, demonstrating that predictive success is not guaranteed even when stereochemistry is explicitly encoded.

Technical Summary: "Carried but Conditionally Used: A Frozen Held-Out Audit of Whether Chirality-Aware Molecular Fingerprints Predict Measured Same-Assay Enantioselectivity"

More like this