Lost in Projection? Gaussian Filtering Recovers Hidden Conformational States

This paper demonstrates that applying Gaussian low-pass filtering to high-dimensional molecular dynamics coordinates effectively mitigates projection artifacts, thereby recovering hidden conformational states and yielding more accurate, long-lived, and structurally defined free energy landscapes.

The Gist

The Big Picture: Trying to See the Forest Through the Trees

Imagine you are trying to understand how a complex machine (like a protein in your body) works. You have a super-fast camera recording every single tiny gear and spring moving inside it. This is what scientists call a Molecular Dynamics (MD) simulation.

The problem? There is too much data. The machine has thousands of moving parts, and looking at all of them at once is overwhelming. To make sense of it, scientists try to squish all that 3D, 4D, or 100D information down into a simple 1D or 2D map. They call this a "projection."

The Analogy: Imagine trying to understand the shape of a complex mountain range by looking only at its shadow cast on a wall.

• The Good: You can see the general peaks and valleys.
• The Bad: The shadow distorts reality. Two separate peaks might look like one flat hill in the shadow. A deep valley might look like a flat plain.

The Problem: "Lost in Projection"

In this paper, the authors explain that when scientists squish their data down to a simple map, they often lose important details.

• The Artifact: Because of the squishing, the map shows that the protein is jumping back and forth between states (like folded and unfolded) way too fast. It looks like the protein is jittering nervously.
• The Consequence: Scientists think the protein is unstable and that its "states" (like a folded shape) are very short-lived. In reality, the protein is stable; the map is just blurry and noisy.
• The Missing State: Sometimes, the squishing is so bad that a whole valley (a stable state) disappears from the shadow entirely. The scientists think the protein never visits that state, but it actually does.

The Old Fix: "Coring" (The "Stay Put" Rule)

Previously, scientists tried to fix this after they made their map. They used a rule called "Coring."

• The Analogy: Imagine you are walking through a foggy forest. You think you've left the forest and entered a field, but you only took one step. The "Coring" rule says: "Don't count it as leaving the forest until you've walked 10 steps into the field without turning back."
• The Flaw: This helps stop the jittering, but it doesn't fix the map itself. If the shadow on the wall made two mountains look like one, telling the walker to "wait 10 steps" doesn't magically make the second mountain appear. You still miss the hidden state.

The New Solution: "Gaussian Filtering" (The Noise-Canceling Headphones)

The authors propose a new method: Gaussian Filtering. Instead of fixing the map after it's made, they clean up the raw data before they make the map.

• The Analogy: Imagine you are listening to a song, but there is a lot of static noise (hiss and crackle) making it hard to hear the melody.
  • Old Way: You try to guess the lyrics while the static is playing.
  • New Way: You put on noise-canceling headphones (Gaussian filtering) that smooth out the static and let the true melody ring through clearly.

By smoothing out the high-frequency "jitter" in the raw movement data, the "shadow" cast on the wall becomes much clearer. The two mountains that looked like one suddenly separate. The hidden valley reappears.

The Results: A Crystal Clear View

The authors tested this on a toy model (a simple math problem) and a real protein (HP35, a tiny piece of a protein that folds quickly).

1. More States Found: When they applied the "noise-canceling" filter to the protein data, they didn't just get a slightly better map. They found 10 times more distinct states (microstates) than before.
2. Better Definitions: The states they found were structurally very clear. They could see exactly how the protein was folded in each state, rather than a blurry mix.
3. Longer Lifetimes: The protein appeared to stay in these stable states for a realistic amount of time, rather than jittering in and out instantly.

Why This Matters

Think of it like upgrading from a grainy, black-and-white security camera to a high-definition 4K camera.

• Before: You could see a person moving, but you couldn't tell if they were holding a bag or a gun, or if they were actually two different people walking past each other.
• After (Gaussian Filtering): The image is so clear you can see the details. You realize there are actually three people, not two, and you can see exactly what they are doing.

The Takeaway

The paper argues that before scientists try to analyze how proteins move, they should first "smooth out" the raw data using this Gaussian filter. It's a simple step that acts like a lens cleaner, revealing hidden states and giving a much more accurate picture of how life's molecular machines actually work. It turns a blurry, confusing shadow into a sharp, detailed map.

Technical Summary

1. Problem Statement

The analysis of Molecular Dynamics (MD) simulations relies on reducing high-dimensional atomic coordinates to low-dimensional collective variables (CVs) to construct Markov State Models (MSMs). This dimensionality reduction often involves projecting data onto suboptimal reaction coordinates, which introduces projection artifacts.

• The Core Issue: Projection artifacts cause the artificial shortening of state lifetimes and, more critically, the complete disappearance of free energy barriers. When high-dimensional data is projected onto a lower dimension, spurious fluctuations occur at transition barriers.
• Consequences:
  • Misclassification: Frames in transition regions are misclassified, leading to rapid, false transitions between states.
  • Loss of States: If a barrier is flattened or lost during projection, distinct metastable states merge, making them undetectable in the analysis.
  • Limitations of Existing Fixes: Current methods like dynamical coring (requiring a system to stay in a state for a minimum time $t_{cor}$) are applied after state definition. While they reduce spurious fluctuations, they cannot recover states that have already vanished from the free energy landscape due to projection errors.

2. Methodology

The authors propose a pre-processing step called Gaussian Filtering applied directly to the high-dimensional input coordinates before dimensionality reduction and clustering.

• Gaussian Low-Pass Filtering:

  • The method applies a Gaussian filter function to the feature trajectory $x(t)$:
    $$x(t) \to \sum_{j} \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left[-\frac{(t_j - t)^2}{2\sigma^2}\right] x(t_j)$$
  • This acts as a low-pass filter with a cutoff frequency determined by the window size $t_{GF} \approx 2\sigma$.
  • Mechanism: It smooths high-frequency fluctuations in the coordinates, effectively narrowing the distance distributions of states and restoring the height of free energy barriers that were artificially lowered by projection.

• Workflow:

  1. Input: Raw MD coordinates (or derived features like contact distances).
  2. Filtering: Apply Gaussian filtering to the feature trajectory.
  3. Dimensionality Reduction: Perform Principal Component Analysis (PCA) on the smoothed data.
  4. Clustering: Use robust density-based clustering (e.g., DBSCAN variants) to identify microstates based on the restored free energy landscape.
  5. MSM Construction: Build the Markov State Model and calculate Implied Timescales (ITS).

• Validation Models:

  1. Toy Model: A 2D three-well potential. The authors compared optimal reaction coordinates against suboptimal projections ($x$-axis and tilted axis $r$) to demonstrate how filtering recovers lost barriers.
  2. Real System: The folding trajectory of the HP35 villin headpiece (300 $\mu$s all-atom simulation). They analyzed inter-residue contacts (42 native contacts) to define the folding process.

3. Key Contributions

• Shift in Paradigm: The paper argues that correcting projection artifacts should be done at the input coordinate level (pre-clustering) rather than the state trajectory level (post-clustering).
• Recovery of Hidden States: Gaussian filtering is shown to recover conformational states that are completely lost in standard projection analyses, a feat unachievable by dynamical coring.
• Optimization of Filtering Window: The authors establish a heuristic for choosing the filtering window $t_{GF}$: it must be long enough to suppress projection noise and recover barriers but short enough to preserve the fastest dynamics of interest.

4. Results

A. Toy Model (2D Three-Well Potential)

• Suboptimal Projection ($x$-axis): The raw projection showed reduced barriers ($\sim 1 k_B T$) and rapid "overshooting" fluctuations.
  • Coring: Reduced spurious transitions but did not restore the barrier height.
  • Gaussian Filtering: Restored the barrier height and clearly separated the three energy basins, allowing for correct state assignment.
• Drastic Projection (Tilted axis $r$): The raw projection merged two wells, resulting in a two-state model where the third state was invisible.
  • Coring: Could not recover the missing third state; the model remained two-state.
  • Gaussian Filtering: Successfully recovered the third minimum in the free energy landscape, allowing the identification of all three states and the correct calculation of two distinct Implied Timescales (ITS).

B. HP35 Folding Trajectory

• Microstate Resolution:
  • Uncorrected Data ($t_{GF}=0$): Identified only 32 microstates.
  • Filtered Data ($t_{GF}=4$ ns): Identified 547 microstates.
  • Filtered Data ($t_{GF}=10$ ns): Identified 990 microstates.
  • Significance: Filtering increased the number of identified microstates by an order of magnitude, revealing a much higher resolution of the free energy landscape.
• Structural Characterization:
  • The uncorrected data merged distinct native-like conformations (differing in Helix 1 details) into a single state.
  • The filtered data ($t_{GF}=4$ ns) resolved these into distinct, structurally well-defined native states.
  • It also better distinguished between fully unfolded states and largely unfolded intermediates with specific intact contacts.
• Dynamics (Implied Timescales):
  • Filtering significantly improved the Markovianity of the model. The first ITS increased, reflecting better separation of timescales.
  • A window of $t_{GF} = 4$ ns was identified as the "sweet spot," yielding correct folding event counts ($\sim 30$ events), converged ITSs, and maximal microstate resolution without over-smoothing.
• Comparison with Dynamical Coring:
  • While dynamical coring ($t_{cor} \approx 4$ ns) produced similar ITSs to filtering, it failed to improve structural resolution. The macrostates derived from coring were structurally identical to the uncorrected data, whereas filtering revealed new structural details.

5. Significance

• Superiority over Coring: The study demonstrates that while dynamical coring improves the kinetics (dynamics) of an MSM, Gaussian filtering improves both the kinetics and the thermodynamics/structure (by recovering hidden states and barriers).
• Standardization: The authors propose that Gaussian filtering of feature trajectories should become a standard component of the MSM workflow. It is fully compatible with existing dimensionality reduction and clustering tools (e.g., PyEmma, MSMBuilder).
• Impact on Biological Insight: By recovering hidden states, this method allows for a more accurate understanding of complex biomolecular processes, such as protein folding pathways, revealing intermediates and transition states that were previously obscured by projection artifacts.

In summary, the paper provides a robust, data-driven solution to a fundamental problem in MD analysis: the loss of information during dimensionality reduction. By smoothing input coordinates, Gaussian filtering restores the underlying free energy landscape, leading to more accurate, high-resolution, and structurally meaningful Markov State Models.