🧬 biology

$\textit{BlockFormer}$ : Transformer-based inference from interaction maps

This paper introduces BlockFormer, a transformer-based method trained on synthetic data that accurately infers parameters from variable-sized interaction maps, demonstrated by its successful application to centromere localization across diverse species.

Original authors: Eloïse Touron, Pedro L. C. Rodrigues, Julyan Arbel, Nelle Varoquaux, Michael Arbel

Published 2026-05-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Eloïse Touron, Pedro L. C. Rodrigues, Julyan Arbel, Nelle Varoquaux, Michael Arbel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you have a giant, complex map that shows how different parts of a city talk to each other. In this paper, the "city" is a living cell, the "parts" are chromosomes (the long strands of DNA that hold our genetic instructions), and the "talk" is physical contact between them. Scientists use a technique called Hi-C to draw this map, which looks like a giant, pixelated heatmap.

The problem is that this map is messy. It's made of many different-sized blocks, and the number of blocks changes depending on the species (a yeast cell has fewer chromosomes than a human). Scientists want to find specific landmarks on this map called centromeres. Think of centromeres as the "waist" of a chromosome; they are crucial for cell division, but finding their exact location on this fuzzy, pixelated map is like trying to find a specific street corner in a foggy city using only a low-resolution satellite photo.

The Old Way: The Slow Detective

Previously, scientists used a method called Centurion. Imagine Centurion as a very thorough but slow detective. To find the waist of a chromosome, it has to:

Guess where the waist might be.
Draw a perfect mathematical curve (a Gaussian shape) over the "fog" on the map.
Adjust that curve, check the math, and repeat the whole process over and over again until it fits perfectly.

This is accurate, but it's incredibly slow. If you want to do this for a new species, the detective has to start from scratch, re-learning the rules every time. It's like hiring a new detective for every single city you visit.

The New Way: BlockFormer (The Smart Translator)

The authors introduce a new tool called BlockFormer. Instead of a detective who solves one case at a time with math, BlockFormer is like a super-smart translator that has read millions of practice maps.

Here is how it works, using simple analogies:

1. The "Lego" Approach (Block-Awareness)
The map isn't a single smooth image; it's a patchwork of different-sized Lego blocks. A standard computer vision tool (like those used for recognizing cats in photos) gets confused because it expects a fixed grid.
BlockFormer is special because it understands that the map is made of blocks. It can look at a tiny block or a huge block and say, "Ah, this is a piece of the puzzle, regardless of its size." It treats each interaction between two chromosomes as a distinct "token" (like a word in a sentence) rather than just a blurry pixel.

2. The "Training Simulator" (The Video Game)
To teach BlockFormer, the authors didn't use real, messy biological data (which is expensive and hard to get). Instead, they built a video game simulator.

They programmed the game to generate thousands of fake maps.
They hid the "waist" (centromere) in random spots.
They made the maps look like the real thing but with a specific "spot" of light where the waist should be.
They trained BlockFormer on these fake maps. Because the game could generate infinite maps instantly, the AI learned to recognize the pattern of the "waist" spot very quickly, even if the map was noisy or the blocks were different sizes.

3. The "Instant Translation"
Once trained, BlockFormer doesn't need to guess and check like the old detective. When you show it a real map from a new species (even one it has never seen before), it instantly "translates" the visual patterns into a precise location.

Speed: It's like the difference between a detective spending 10 hours solving a crime and a security camera recognizing a face in 0.1 seconds.
Flexibility: It works on yeast, parasites, and plants without needing to be retrained. It just looks at the map and says, "I see the pattern; the waist is here."

What Did They Find?

The paper tested this on real biological data from various species (yeasts, parasites, and plants).

Accuracy: BlockFormer found the centromeres just as accurately as the slow, old method (and sometimes better), often pinpointing the location with a precision finer than the map's own resolution.
Speed: It was dramatically faster. In some cases, it was hundreds of times faster than the old method.
Robustness: Even when the maps were very noisy (like a photo taken in a storm) or had weird shapes, BlockFormer kept working well, whereas the old method often got confused or failed.

The Bottom Line

The paper presents BlockFormer as a new, fast, and flexible way to find specific landmarks on complex genetic maps. By treating the map as a collection of variable-sized blocks and training on a massive library of simulated examples, the AI can instantly identify biological features that previously required hours of slow, manual calculation. It turns a difficult, one-off math problem into a quick, automated recognition task.

Technical Summary: BlockFormer – Transformer-based Inference from Interaction Maps

1. Problem Definition

The paper addresses a generic inverse problem: inferring per-entity parameters from an interaction map that summarizes pairwise relationships between entities. These maps often exhibit a block-wise structure, where each block represents interactions between two specific entities. Crucially, the number and size of these blocks vary significantly across different instances (e.g., different species or experimental resolutions), and the maps contain both localized patterns within blocks and non-local alignment information across blocks.

A primary biological application highlighted is centromere localization from Hi-C contact maps. In this context, the interacting entities are chromosomes, and the goal is to identify the genomic position of centromeres ( $\theta$ ) based on interaction peaks in the trans-blocks (inter-chromosomal interactions). Existing methods, such as Centurion, rely on fitting Gaussian profiles to interaction peaks via non-convex optimization. While effective, these approaches are computationally expensive, non-amortized (requiring re-optimization for every new dataset), and sensitive to initialization. Furthermore, standard deep learning approaches (CNNs, Graph Neural Networks) struggle with the variable dimensions and block-wise organization of these maps.

2. Methodology: BlockFormer

The authors propose BlockFormer, a transformer-based architecture designed specifically to handle interaction maps with variable block numbers and sizes.

Architecture

Block-Aware Tokenization: The input map is decomposed into sequences of blocks. To respect the block structure, the authors apply asymmetric zero-padding to each block to ensure dimensions are multiples of a patch size ( $P$ ). The map is then cut into squared patches.
3D Positional Encoding: A key innovation is the use of a 3D positional encoding $(i, j, k)$ $(i, j, k)$ for each patch, where:
- $i$ is the block index.
- $(j, k)$ is the relative position of the patch within that block.
  This encoding allows the model to distinguish between different blocks and retain spatial information within blocks, enabling generalization to varying numbers of entities and block sizes.
Transformer Backbone: The architecture follows the Vision Transformer (ViT) paradigm, alternating Multi-head Self-Attention, LayerNorm, and MLP blocks. A learnable class token is prepended to the sequence to aggregate global information and output the parameter estimate.
Output Projection: The class token is projected via a linear layer and sigmoid activation to produce a normalized estimate of the parameter (e.g., relative position within a chromosome), which is then rescaled to the actual genomic coordinate.

Training Strategy and Simulation

Simulation-Based Inference (SBI): To avoid the cost of manual annotation and the complexity of biophysical simulators, the authors designed a lightweight contact map simulator. This simulator generates synthetic interaction maps by placing Gaussian "spots" at centromere positions within trans-blocks, adding noise, and simulating rare interactions.
Flexible Training: The model is trained on batches of synthetic maps with variable numbers of blocks (chromosomes) and variable block sizes. This strategy forces the network to learn representations invariant to the specific number and size of entities, enabling it to generalize to unseen species and resolutions.
Loss Function: The model is trained using a standard regression loss (Mean Squared Error) between the predicted parameter and the ground truth from the simulator.

Uncertainty Quantification

Beyond point estimation, BlockFormer serves as a summary statistic within a Bayesian framework. The authors integrate the trained model into Approximate Bayesian Computation (ABC) and Neural Posterior Estimation (NPE) pipelines to quantify uncertainty in parameter estimates, providing posterior densities rather than just point estimates.

3. Key Contributions

BlockFormer Architecture: A novel transformer architecture that explicitly models the block-wise structure of interaction maps through per-block padding and 3D positional encoding, solving the challenge of variable input dimensions.
Data-Driven Approach: The development of a simplified, computationally efficient simulator that captures the essential structural patterns (spot-like interactions) required for inference, bypassing the need for expensive biophysical simulations or manual annotation.
Amortized Inference: Unlike optimization-based methods (e.g., Centurion), BlockFormer is amortized; once trained, it performs inference via a single forward pass, making it significantly faster and applicable to diverse datasets without re-training.
Robustness to Variability: The method demonstrates robustness to varying numbers of entities, block sizes, sequencing depths, and noise patterns, achieving sub-resolution precision in many cases.

4. Results

The authors evaluated BlockFormer on real-world Hi-C data across seven yeast species, a parasite (P. falciparum), and a plant (A. thaliana), as well as on synthetic data.

Accuracy: BlockFormer achieves sub-resolution precision (errors below the map resolution, e.g., <32 kb) across a wide range of species and genome sizes. In many cases, it matches or outperforms the state-of-the-art Centurion method in accuracy.
Efficiency: The method is drastically faster. For example, on P. falciparum, BlockFormer achieved an error of 0.27 in 24.6 seconds, whereas Centurion required 185.5 seconds for a similar error. The amortized nature of the model means inference time scales linearly with data size, independent of the optimization iterations required by Centurion.
Generalization: The model successfully generalizes to species with chromosome counts and sizes outside the training distribution (e.g., A. thaliana with large chromosomes) and to different sequencing depths (10% to 50%).
Robustness to Noise: BlockFormer outperforms Centurion in scenarios with noisy maps, multiple spots per block, or non-Gaussian spot shapes (e.g., square or ring spots), where Centurion's Gaussian fitting assumptions fail.
Posterior Estimation: When used as a summary statistic in Bayesian inference, BlockFormer-based approaches (ABC-Transf, NPE-Transf) produce sharper, better-calibrated posterior distributions compared to CNN-based alternatives.
Extension to Loop Localization: The authors demonstrated that the approach can be adapted to localize DNA loops (intra-chromosomal features), outperforming Centurion in both speed and accuracy on single-block inference tasks, provided a pre-localization step is used.

5. Significance and Claims

The paper claims that BlockFormer provides a flexible, amortized, and data-driven solution for inferring parameters from interaction maps with variable structures. Its significance lies in:

Overcoming Structural Constraints: It successfully handles the variable number and size of interaction blocks, a limitation of standard CNNs and GNNs.
Bypassing Optimization Bottlenecks: By replacing iterative non-convex optimization with a single forward pass, it enables rapid inference on large-scale genomic datasets.
Handling Real-World Complexity: The method remains robust even when the simulator does not perfectly match the complex biophysics of real data, suggesting that capturing key structural patterns is sufficient for accurate inference.
Uncertainty Quantification: It facilitates principled uncertainty quantification by serving as an effective summary statistic for simulation-based inference.

The authors modestly note limitations: the model outputs a single parameter per entity, which limits applicability when the number of parameters is unknown or when maps contain no parameter information (e.g., loop localization requires pre-filtering to single-loop regions). They suggest future work could apply this framework to other inference tasks, such as inferring 3D genome configurations, using more biologically inspired simulators.