DeepRHP: A Hybrid Variational Autoencoder for Designing Random Heteropolymers as Protein Mimics

Technical Summary: DeepRHP

Problem Statement
Synthetic random heteropolymers (RHPs) offer a pathway to designing protein-like materials capable of stabilizing proteins in non-native environments and facilitating functions like proton transport. However, designing RHPs with specific desired functions is challenging because synthetic RHP chains lack deterministic sequences and conformations. Unlike traditional protein design, which relies on high-throughput sequencing and 3D structures for methods like directed evolution or de novo design, RHP design currently lacks rational principles. Existing approaches are largely empirical, relying on time-intensive laboratory screenings of various monomer compositions and chain lengths. Furthermore, there is a scarcity of computational literature addressing how to model RHPs as an ensemble rather than individual chains, leaving critical design questions unanswered: specifically, how many monomers are required in a system's "alphabet," and how to identify monomer compositions that correspond to specific protein functions.

Methodology: DeepRHP
To address these gaps, the authors propose DeepRHP, a modified Variational Autoencoder (VAE) trained under a semi-supervised framework. The model is designed to learn low-dimensional latent representations of RHP sequences that capture both sequence patterns and critical chemical features.

• Hybrid Architecture: Unlike standard VAEs that concatenate features into a single vector, DeepRHP employs a hybrid architecture consisting of a classical VAE and an additional feature-driven VAE operating in parallel. Both share a common latent variable $z$.
• Semi-Supervised Learning: The model utilizes sequence data $x$ and function-related chemical features $y$ (specifically, the average hydrophilic–lipophilic balance (HLB) of sliding windows). Since $y$ is a deterministic transformation of $x$, the encoder relies solely on $x$, while the decoder is equipped to reconstruct both the sequence and the feature.
• Objective Function: The training objective maximizes the log-likelihood of the sequence by optimizing a modified Evidence Lower Bound (ELBO). This bound includes a weighted sum of the reconstruction losses for both the sequence ($x$) and the chemical feature ($y$), alongside the Kullback-Leibler (KL) divergence term:
  $$\log p(x) \geq \alpha E_q [\log p(x | z)] + (1 - \alpha) E_q [\log p(y | z)] - D_{KL} (q(z | x) \parallel p(z))$$
  Here, $\alpha$ is a hyperparameter balancing the weight between sequence structure and chemical feature fidelity.
• Implementation: The encoder and decoder are implemented as multilayer perceptrons (MLPs) using PyTorch. The model is trained using the ADAM optimizer.

Key Contributions

1. First Deep Learning Application for RHP Design: This work is the first to utilize deep learning to answer fundamental RHP design questions, learning interpretable latent representations for RHP ensembles.
2. Insight into Design Parameters: The model provides data-driven insights into two critical parameters: monomer alphabet size and monomer composition.
3. Flexibility and Interpretability: The hybrid architecture allows for the incorporation of any relevant chemical features, making the method versatile for various protein functions. Furthermore, by linking latent variables directly to chemical features, the model offers interpretability often lacking in classical VAEs, where principal directions are physically obscure.

Results
The authors validated DeepRHP using a system of four methacrylate-based monomers (MMA, EHMA, OEGMA, SPMA) and 30,000 membrane and globular protein sequences from UniProt.

• Monomer Composition Validation: The model successfully validated the empirical finding that a 30% hydrophilic to 70% hydrophobic monomer ratio is optimal for stabilizing membrane proteins like Aquaporin Z (AqpZ). Principal Component Analysis (PCA) of the latent space showed that RHP compositions with this ratio (specifically RHP B and C in a two-monomer system) overlapped significantly with natural membrane and globular proteins.
• Alphabet Size Analysis: The study demonstrated that a two-monomer alphabet creates a sequence space that is too broad relative to specific proteins. In contrast, expanding to a four-monomer system (varying the MMA:EHMA ratio while maintaining the 30:70 hydrophilic/hydrophobic balance) resulted in a more localized sequence space. The latent projections of AqpZ in the four-monomer space almost entirely covered the spaces of RHP 4 and RHP 5.
• Experimental Concordance: The model predicted that RHP 4 and RHP 5 compositions would be optimal. This prediction aligned with published wet-lab experiments (Panganiban et al. 2018), which identified an RHP with the same monomer ratio as RHP 4 as the most effective stabilizer for AqpZ.
• Ablation: In ablation studies, the hybrid DeepRHP model outperformed a singular classical VAE without the additional decoding regressor.

Significance and Claims
The paper claims that DeepRHP demonstrates the potential of utilizing hybrid autoencoder architectures to guide the design of RHPs for proteins and other biological compounds. By bridging the gap between empirical screening and rational design, the tool offers a platform for similarity analysis between target proteins and RHP sequences. The authors position this work as a "first step" in integrating deep learning into macromolecular cheminformatics. They maintain a modest tone regarding future work, noting that current assessments are qualitative and validated against laboratory results. They propose that future efforts should focus on developing quantitative measures to evaluate the quality of latent representations and performing downstream tasks such as classifying specific membrane proteins.