Evolutionary conditioning enables guided generation of functionally diverse enhancers

1. Problem Statement

The design of synthetic enhancers (DNA sequences that regulate gene expression) is a critical challenge in synthetic biology and gene therapy. Current deep learning approaches for enhancer design typically rely on cell-type label conditioning. These models require massive amounts of labeled data from Massively Parallel Reporter Assays (MPRAs) to learn the relationship between DNA sequence and activity in specific cell types.

• Limitations: This approach is constrained by the scarcity of MPRA data (available for only a handful of cell types) and biases models toward experimentally tractable cell lines, failing to address transient or difficult-to-isolate developmental cell types.
• Goal: The authors aim to develop a generative model that can design functional enhancers for specific contexts without requiring explicit cell-type labels or extensive assay data, leveraging evolutionary conservation instead.

2. Methodology: EnhancAR

The authors propose EnhancAR, an autoregressive generative model trained on evolutionary homologs rather than labeled assay data.

• Data Construction:
  • Source: 1.7 million candidate human enhancers from ENCODE SCREEN (spanning 1,888 cell types).
  • Homolog Extraction: Using the Zoonomia 241-species whole-genome alignment, they extracted homologous sequences for each human enhancer, resulting in ~233 million homologs (average ~135 per enhancer).
  • Dataset: 1.7 million unaligned enhancer homolog families.
• Model Architecture:
  • Base: A Jamba model (a hybrid Transformer/State-Space model), chosen for its ability to handle long context windows with lower memory usage than standard Transformers.
  • Training Input: Homolog families are "unrolled" into a single string of sequences separated by special tokens (e.g., {Seq1/Seq2/Seq3}).
  • Training Objective: Next-token prediction (autoregressive) using cross-entropy loss. The model learns the distribution of sequences within a homolog family.
  • Variants:
    • EnhancAR: Trained on randomly ordered homologs.
    • EnhancAR-sorted: Trained on homologs sorted by descending length to enable sequence shortening.
• Inference Strategy (Prompting):
  • Instead of conditioning on a cell-type label, the model is prompted with a set of homologous sequences from a specific enhancer family.
  • The model generates new sequences that statistically resemble the prompt set, effectively "evolving" new variants that preserve the conserved functional properties of the prompt.

3. Key Contributions

1. Novel Conditioning Paradigm: Shifts from cell-type label conditioning to evolutionary conditioning (homolog prompting), enabling design for contexts where labeled data is absent.
2. Largest Homolog Dataset: Curated the largest dataset of unaligned enhancer homolog families to date (1.7 million families).
3. Functional Preservation: Demonstrated that the model can generate novel sequences that preserve motif composition and predicted regulatory activity without explicit functional labels during training.
4. Sequence Shortening: Introduced a method to design shorter enhancers (critical for viral vector delivery) by training on length-sorted homologs, successfully reducing sequence length while maintaining activity.

4. Key Results

A. Unconditional Generation Resembles Natural Homologs

• When generating sequences without prompts, EnhancAR produced sets that were distinct from the training data (low BLASTN overlap) but shared statistical properties with real homolog families.
• Sequence Diversity: Generated sets had lower mean pairwise identity (65%) than real homologs (73%) but significantly higher than scrambled sequences (~57%), indicating the model learned evolutionary constraints.
• Motif Conservation: Generated sets preserved enriched transcription factor (TF) motifs found in real homologs, as measured by Average Motif Scores (AMS).
• Functional Space: Embeddings from Enformer (a predictive model) showed that generated sequences occupy a similar functional space to real enhancers.

B. Prompted Generation Preserves Specificity

• Novelty & Diversity: When prompted with a specific homolog family, the model generated sequences that were novel (lower similarity to the prompt than natural human homologs) but maintained the diversity profile of the prompt.
• Motif Specificity: The model successfully preserved the strongest motifs of the prompt. For example, when prompted with POU5F1 enhancer homologs, the generated sequences retained the POU5F1::SOX2 co-motif with high precision (0.83) and recall (0.74).
• Cell-Type Specificity without Labels: By prompting with homologs of known cell-type-specific enhancers (e.g., HepG2, K562, WTC11), the model generated sequences that showed predicted cell-type specificity using MPRALegNet models.
  • Example: HepG2-prompted generations showed high activity in HepG2 models but low activity in K562/WTC11 models, despite the model never seeing cell-type labels.
• Prompt Size Effect: Increasing the number of sequences in the prompt (up to 63) improved the predicted activity and motif enrichment of the generated designs, suggesting the model leverages deeper evolutionary information.

C. Enhancer Shortening (EnhancAR-sorted)

• By training on length-sorted homologs, the model learned to generate sequences shorter than the prompt.
• Results: 85% of generated sequences were shorter than the shortest prompt sequence.
• Functionality: 38–43% of these shortened designs maintained high predicted activity (>2 log2 FC) in the target cell type.
• Case Study: A K562 enhancer was successfully shortened from 200 bp to 53 bp while retaining a strong GATA1::TAL1 motif and high predicted activity.

5. Significance and Future Directions

• Generalizability: EnhancAR offers a flexible paradigm for designing enhancers for "orphan" cell types (e.g., transient developmental stages) where MPRA data does not exist, provided homologous sequences can be identified.
• Therapeutic Utility: The ability to shorten enhancers while preserving function addresses a major bottleneck in gene therapy (viral vector capacity limits).
• Mechanistic Insight: The model acts as a tool to probe the cis-regulatory code. By analyzing attention mechanisms and motif conservation, it can help identify higher-order rules (e.g., motif spacing, orientation) governing enhancer function.
• Limitations: The model's output is bounded by the function of the prompt's homologs; if the homologs have lost function (evolutionary drift), the generated sequences may reflect that. Future work suggests incorporating phylogenetic trees or species tokens to better control evolutionary distance.

In summary, EnhancAR demonstrates that evolutionary information encoded in homologous sequences is a powerful, label-free signal for training generative models to design functional, diverse, and compact enhancers.