HitAnno: Atlas-level cell type annotation based on scATAC-seq data via a hierarchical language model

HitAnno is a hierarchical language model that leverages cell-type-specific peaks and a two-level attention mechanism to provide accurate, scalable, and interpretable cell type annotation for atlas-level scATAC-seq data across diverse settings without requiring retraining.

The Gist

Imagine your body is a bustling, massive city with millions of different neighborhoods (cells). Each neighborhood has its own unique "rulebook" (DNA) that tells the buildings (genes) how to operate. Sometimes, these rulebooks have sections that are "open for business" (accessible) and sections that are "closed for renovation" (inaccessible).

Scientists use a technology called scATAC-seq to take a snapshot of these open and closed sections for every single cell. It's like trying to read a library of millions of books, but the pages are torn, the ink is faint, and the books are written in a strange code.

The big challenge? Figuring out exactly what kind of neighborhood each cell belongs to just by looking at these torn pages. Is this a "Neuron" neighborhood? A "Liver" neighborhood? Or a rare "Immune Cell" neighborhood?

Currently, doing this is like trying to sort a million mixed-up puzzle pieces by hand. It's slow, prone to human error, and gets messy when the puzzle is huge or the pieces are very similar.

Enter HitAnno.

The Big Idea: Treating Cells Like Sentences

The researchers behind HitAnno had a brilliant insight: What if we treat a cell's genetic data like a sentence in a language?

• The Words (Tokens): Instead of words, the "words" are specific spots on the DNA where the chromatin is open.
• The Sentence: A whole cell is a long, complex sentence made up of these open spots.
• The Grammar: The way these spots are arranged tells us the cell's identity.

How HitAnno Works: The Two-Level Translator

HitAnno is a smart computer program (a "Hierarchical Language Model") that reads these genetic sentences. To make it fast and accurate, it uses a two-level attention system, which is like a translator who works in two steps:

1. Level 1: The Clause Reader (Local Attention)
   Imagine a sentence has different clauses, like "The cat sat on the mat" and "The dog barked."
   HitAnno first looks at small groups of DNA spots (clauses) that are specific to certain cell types. It asks: "Do these specific spots usually appear together in a 'Neuron' sentence?" It captures the local details, like how the words in a specific phrase relate to each other.

2. Level 2: The Whole Story Reader (Global Attention)
   Once it understands the clauses, it steps back to look at the whole sentence. It asks: "Given all these clauses, what is the main topic of this story?"
   This helps it decide if the cell is a Neuron, a Liver cell, or something rare. It connects the dots between different parts of the genome to make a final, confident guess.

Why Is HitAnno Special?

1. It's a Master at Rare Finds
Most old methods are like a teacher who only pays attention to the students raising their hands the loudest (the common cell types). They often miss the quiet students in the back (rare cell types). HitAnno, however, is like a teacher who listens to everyone. It was trained to recognize even the rarest cell types, ensuring no one gets left out of the census.

2. It Handles the "Noise" of Real Life
In the real world, data is messy. One dataset might be taken with a different camera than another, or from a different person. Old methods get confused by these differences. HitAnno is like a polyglot who can understand the same story even if it's told in different accents or dialects. It ignores the "noise" (batch effects) and focuses on the core meaning (the cell type).

3. It's an Atlas-Scale Super-Tool
Usually, if you want to analyze a new dataset, you have to retrain the computer model from scratch. That takes forever.
HitAnno is different. The researchers trained it on a massive "Atlas" of 31 different cell types from humans. Once trained, it's like a universal translator. You can feed it any new dataset, and it can instantly label the cells without needing to be retrained. It's a "one-and-done" model that keeps working forever.

4. It's Explainable (No Black Box)
Many AI models are "black boxes"—they give an answer, but you don't know why. HitAnno is transparent. Because it uses an "attention" mechanism, it can show you which parts of the DNA sentence it focused on to make its decision.

• Analogy: If HitAnno says "This is a Liver cell," it can point to the specific DNA spots it looked at and say, "I knew it was a liver cell because these specific spots were open, just like they are in all other liver cells." This helps scientists trust the result and even discover new biological rules.

The Result

The researchers built a website where anyone can upload their genetic data, and HitAnno will instantly sort the cells for them. They tested it on huge, complex datasets (like the entire human brain) and found it was more accurate and reliable than any previous method.

In short: HitAnno turns the chaotic, messy language of our cells into a clear, organized story, helping scientists understand how our bodies are built and how diseases might start, all while being fast, accurate, and easy to use.

Technical Summary

1. Problem Statement

Single-cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq) is a critical technology for understanding epigenomic heterogeneity and gene regulatory programs. However, accurate cell type annotation of scATAC-seq data faces significant challenges, particularly as datasets scale to "atlas-level" (large-scale, diverse, and complex):

• Data Characteristics: scATAC-seq data is high-dimensional, extremely sparse, and near-binary, making it difficult for standard models to extract robust features.
• Scalability and Bias: Existing supervised methods struggle with the massive cell numbers in atlas datasets. They often suffer from class imbalance bias, where models are dominated by major cell types and fail to distinguish rare populations.
• Generalization: Current methods often lack robustness when applied to unseen donors or datasets (cross-donor/inter-dataset annotation) due to batch effects and technical heterogeneity.
• Interpretability: Many deep learning models act as "black boxes," failing to provide biologically interpretable insights into the chromatin accessibility patterns driving annotation decisions.

2. Methodology: HitAnno

HitAnno is a supervised method that frames the chromatin accessibility landscape as a hierarchical language. It utilizes a Transformer-based architecture to process "cell sentences" constructed from scATAC-seq data.

Core Architecture

The model consists of three modules:

1. Tokenization Module (Cell Sentence Construction):

   • Concept: A cell is represented as a "sentence" composed of multiple "clauses."
   • Peak Selection: For each cell type in the reference, a specific set of peaks (a specialized vocabulary) is identified using TF-IDF transformation and statistical testing (Welch's t-test).
   • Structure: Each cell sentence is formed by concatenating clauses. Each clause corresponds to a specific cell type and contains a sequence of binarized accessibility values for that cell type's specific peaks.
   • Tokens: A special [CLS] token marks the start of each clause (cell-type specific), and a global [CLS] token represents the entire cell. All clauses are padded to equal length to prevent the model from biasing toward major cell types.

2. Representation Module (Hierarchical Transformer):

   • Input Embedding: Combines embeddings of binary accessibility values and peak indices.
   • Two-Level Attention Mechanism:
     • Peak-Level Transformer: Processes each clause independently in parallel. It captures local co-accessibility patterns among peaks within a specific cell-type-specific peak set. It outputs an embedding for the [CLS] token of that clause.
     • Cell-Type-Level Transformer: Takes the collection of clause embeddings (plus the global [CLS]) and captures global dependencies between different cell types (peak sets). This produces the final cell embedding.
   • Design Philosophy: This "divide-and-conquer" approach allows the model to scale efficiently to large atlases while explicitly modeling interactions at both local (peak) and global (cell-type) levels.

3. Annotation Module:

   • A Multi-Layer Perceptron (MLP) classifier takes the final cell embedding and outputs a probability distribution over cell types.

Training Strategy

• Trained on a unified atlas-scale reference dataset (31 cell types) integrating adult brain and fetal scATAC-seq data.
• Once trained, the model can annotate new query datasets without retraining, acting as a reusable foundation model.

3. Key Contributions

• Hierarchical Language Modeling: First application of a hierarchical Transformer architecture to scATAC-seq, treating accessibility profiles as structured sentences with cell-type-specific clauses.
• Atlas-Level Scalability: Successfully trained on a 31-cell-type atlas and demonstrated the ability to directly annotate new, unseen datasets without retraining.
• Robustness to Imbalance: Explicitly designed to handle rare cell types by enforcing equal clause lengths and using cell-type-specific peak sets, preventing dominance by major populations.
• Interpretability: The hierarchical attention mechanism provides biological insights:
  • Peak-level attention correlates with known co-accessibility patterns (validated by Cicero and Hi-C data).
  • Peak-set-level attention reveals dependencies between cell types, showing the model focuses on discriminative peak sets corresponding to true cell identities.
• Online Deployment: Packaged as a user-friendly web tool for automated annotation.

4. Results

The authors evaluated HitAnno across 12 diverse scATAC-seq datasets and multiple challenging scenarios:

• Intra-dataset Performance:

  • Outperformed five state-of-the-art baselines (SANGO, Cellcano, EpiAnno, MLP, PCA+SVM) across 12 datasets.
  • Achieved an average accuracy of 89.74% (6.91% higher than the second-best method, SANGO).
  • Showed superior performance on macro-F1 and Cohen's kappa, indicating better discrimination of rare cell types.
  • Demonstrated robustness under extreme cell type imbalance, simulated dropout, and downsampling.

• Cross-Donor and Inter-dataset Generalization:

  • In cross-donor experiments, HitAnno maintained high accuracy despite batch effects, outperforming baselines by up to 9.51% in macro-F1.
  • In inter-dataset tasks (e.g., annotating fetal brain data using adult references), HitAnno consistently achieved the best performance across all metrics, whereas baselines showed high variability.
  • Successfully resolved ambiguous cell populations (e.g., distinguishing astrocytes from microglia in mixed glial populations) where other methods failed.

• Atlas-Level Application:

  • When applied to external datasets (Kanemaru2023, Li2023b, Domcke2020_thymus) using the pre-trained 31-cell-type model, HitAnno achieved accuracies of 89.63%, 82.94%, and 93.48% respectively.
  • Correctly grouped related subtypes (e.g., atrial/ventricular cardiomyocytes) under broader categories, demonstrating hierarchical understanding.

• Interpretability Validation:

  • Peak-level attention maps aligned with Cicero co-accessibility predictions and Hi-C contact maps, confirming the model learns biologically meaningful local interactions.
  • Peak-set-level attention correctly concentrated weights on the peak sets corresponding to the true cell type, validating the global decision-making process.

5. Significance

• Paradigm Shift: HitAnno moves scATAC-seq annotation from manual, cluster-based labeling to a scalable, automated, and interpretable deep learning framework.
• Foundation Model Potential: By training on a comprehensive atlas and deploying without retraining, HitAnno acts as a foundational model for the field, capable of handling the growing volume and diversity of single-cell epigenomic data.
• Biological Insight: The model's ability to interpret attention weights provides a new avenue for discovering regulatory relationships and co-accessibility patterns directly from the annotation process.
• Practical Utility: The availability of an online interface makes advanced cell type annotation accessible to researchers without deep learning expertise, facilitating the refinement of existing cell atlases and the discovery of novel subpopulations.