🧬 biology

HiFi-ST: High-Fidelity Reconstruction of Continuous Spatial Transcriptomic Expression Fields via Conditional Neural Fields

HiFi-ST introduces a conditional neural field framework that reconstructs continuous spatial transcriptomic expression fields by modeling spots as regional observations and integrating multiscale tissue features, thereby significantly outperforming existing methods in accuracy and enabling advanced downstream analyses like tumor microenvironment characterization.

Original authors: Lei Tang, Wenshuai Han, Xiao Yang, Xiaozhou Chen, Huamei Li

Published 2026-07-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lei Tang, Wenshuai Han, Xiao Yang, Xiaozhou Chen, Huamei Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to understand the weather in a massive, bustling city. You have a few weather stations scattered across the map, each giving you a temperature reading for its specific spot. But the city isn't just a collection of isolated points; the air flows continuously, and the temperature changes smoothly as you walk from a park to a skyscraper. If you only look at the weather stations, you might miss the gentle breeze in the alleyways or the heat rising off the pavement between the sensors. This is exactly the challenge scientists face when studying the "city" inside a living body: the tissue.

In biology, tissues are made of cells that talk to each other using chemical messages called genes. For a long time, scientists could only take "snapshots" of these messages at specific, scattered locations, like those weather stations. This method, called spatial transcriptomics, is great, but it leaves huge gaps. It's like trying to paint a masterpiece using only a few dots of color; you can guess the picture, but you miss the smooth lines and the subtle blending of colors. The big question is: How do we fill in the blanks to see the full, continuous picture of how genes behave across an entire tissue, rather than just at the spots where we happened to look?

Enter HiFi-ST, a new tool developed by researchers that acts like a super-smart "fill-in-the-blanks" artist for biology. Instead of just guessing the gene activity at the exact spots where samples were taken, HiFi-ST treats the tissue as a continuous, flowing field of information. Think of it as upgrading from a pixelated, low-resolution image to a high-definition, smooth video. The researchers built a system that learns from the tissue's visual appearance (what it looks like under a microscope) and the scattered gene data to reconstruct the entire "gene landscape" in between the dots.

Here is how they did it and what they found. Traditional methods tried to predict gene activity by treating each sample spot as an isolated point, like trying to guess the temperature of a whole room based on a single thermometer. The authors argue this misses the bigger picture because biology is continuous. HiFi-ST changes the game by using something called a "neural field." Imagine this as a magical canvas that doesn't just know the colors at specific dots but understands how colors blend and flow across the whole surface.

To make this work, the model looks at the tissue at three different "zoom levels" simultaneously: a close-up, a medium view, and a wide view. It's like looking at a forest through a magnifying glass to see individual leaves, then stepping back to see a cluster of trees, and finally zooming out to see the whole forest. By combining these views, the model understands the complex shapes and structures of the tissue. It then uses a clever trick called "Monte Carlo sampling." Instead of guessing the gene activity at one single point, the model takes 16 tiny, random "sniffs" around each location and averages them. This is similar to how a chef might taste a soup from different parts of the pot to get the perfect flavor, rather than just tasting the very top. This helps the model handle the messy, uneven nature of real tissue samples.

The results are quite impressive. The researchers tested HiFi-ST on three different types of tissue datasets: breast cancer (HER2+), skin cancer (cSCC), and a general dataset called Alex_NatGen. In the breast cancer dataset, HiFi-ST improved the accuracy of matching gene patterns by 65.1% compared to the previous best methods, while reducing the error in gene counts by 40.9%. In the skin cancer dataset, it reduced errors by 51.2%, and in the general dataset, it improved pattern matching by a massive 80.0%.

But the magic doesn't stop at just making better pictures. Because HiFi-ST creates a smooth, continuous map of gene activity, it can help scientists spot hidden structures that were previously hard to see. For example, the model helped identify "Tertiary Lymphoid Structures" (TLS), which are like tiny, organized immune fortresses that form inside tumors. The model successfully flagged candidate areas for these structures, suggesting it could help doctors understand how the immune system is fighting a tumor, even in areas where no specific sample was taken.

The authors are careful to note that while this is a significant step forward, it's not a perfect solution yet. The model currently works on flat, 2D slices of tissue and doesn't yet account for the full 3D shape of organs. It also relies on some assumptions about how the tissue is sampled that might not be perfect in every single biological scenario. However, by bridging the gap between scattered data points and a continuous, high-fidelity view of gene expression, HiFi-ST offers a powerful new way to explore the complex, flowing world inside our bodies. It suggests that by treating biology as a continuous field rather than a collection of dots, we can uncover patterns and details that were previously hidden in the gaps.

Technical Summary of HiFi-ST: High-Fidelity Reconstruction of Continuous Spatial Transcriptomic Expression Fields via Conditional Neural Fields

Problem Statement
Spatial transcriptomics (ST) technologies characterize tissue-scale gene expression patterns but are fundamentally limited by their discrete sampling mechanism. Due to physical capture-array spacing, ST observations represent sparse, discrete samples of an underlying continuous molecular field. This discretization leads to spatial aliasing and the loss of sub-resolution information. Furthermore, existing computational methods for predicting spatial gene expression from histological images (H&E) typically formulate the task as "spot-level point regression." This approach fails to explicitly model the intrinsic continuity of gene expression or the regional nature of the physical measurement process, where each spot captures an aggregate signal over a finite support domain rather than a single point.

Methodology: The HiFi-ST Framework
The authors propose HiFi-ST, a conditional neural field framework designed to bridge the gap between discrete measurements and continuous expression field reconstruction. The framework operates through four primary stages:

Data Preprocessing and Conditioning:
- The model takes multiscale H&E image patches and spatial coordinates as input.
- Highly Variable Genes (HVGs) are selected, and expression matrices are normalized (CPM-like) and log-transformed.
- To capture global expression structure, the first 8 principal components (PCA) of the expression matrix are computed offline to serve as a low-dimensional expression embedding.
- A shared global condition vector is constructed by fusing the spatial coordinates (via Fourier positional encoding) and the expression PCA embedding.
Multiscale Feature Extraction:
- To capture tissue morphology at varying scales, the model extracts three image patches around each spatial location: 112×112, 224×224, and 448×448 pixels.
- Each scale is processed by a dedicated convolutional neural network (CNN) encoder to generate visual feature representations.
Conditional Modulation and Fusion:
- The framework employs Feature-wise Linear Modulation (FiLM) to inject contextual priors into the visual features. The global condition vector is mapped to scaling ( $\gamma$ ) and shifting ( $\beta$ ) parameters for each scale, allowing the model to adapt visual features based on the specific tissue context.
- A multiscale fusion module aggregates the modulated features using learnable scale weights.
Prediction via Monte Carlo Integration:
- Unlike point-regression models, HiFi-ST treats each spot as a regional observation. To approximate the expected regional expression, the model performs Monte Carlo sampling.
- For each spatial location, $M$ (default 16) perturbed coordinates are sampled from an isotropic Gaussian distribution.
- The neural field predicts expression values at these sampled coordinates, and the final spot-level prediction is obtained by averaging these samples. This strategy approximates local integration and improves robustness to spatial noise and registration errors.

Key Contributions
The paper outlines three primary contributions:

Paradigm Shift: Reformulating spatial gene expression prediction from discrete point-wise regression to continuous expression field learning.
Measurement Consistency: Explicitly introducing regional observation modeling to better align with the physical measurement mechanism of ST technologies, utilizing Monte Carlo sampling to approximate local integration.
Architectural Integration: Combining multiscale conditional modulation (via FiLM) with conditional neural field decoding to effectively model complex spatial heterogeneity and reconstruct high-fidelity expression patterns in complex tissues.

Experimental Results
HiFi-ST was evaluated on three independent datasets: HER2+ (breast cancer), cSCC (cutaneous squamous cell carcinoma), and Alex_NatGen (general tumor microenvironment). It was compared against five state-of-the-art baselines: HisToGene, His2ST, THItoGene, BLEEP, and mclSTExp.

Quantitative Performance: HiFi-ST consistently outperformed baselines across key metrics (PCC, HEG_PCC, MSE, MAE).
- On HER2+, it achieved a 65.1% improvement in average PCC and a 40.9% reduction in MSE compared to the best baseline.
- On cSCC, PCC improved by 10.2% and MSE decreased by 51.2%.
- On Alex_NatGen, PCC improved by 80.0% and MSE decreased by 16.3%.
Spatial Pattern Recovery: The model demonstrated superior ability to recover fine-grained spatial patterns for individual genes, showing higher median correlations and identifying more high-confidence genes (e.g., FN1, IGKC, HLA-DRA) consistent with ground truth.
Downstream Utility: The learned multiscale representations were successfully applied to assist in identifying candidate Tertiary Lymphoid Structure (TLS) regions, achieving high recall (mean 0.68) in candidate prioritization.
Robustness: In the cSCC cohort, characterized by severe morphological variation (keratin pearls, nonlinear deformation), HiFi-ST maintained stable reconstruction of key genes (FTL, SPRR2A) where baselines struggled.

Significance and Claims
The authors claim that HiFi-ST provides a unified framework that elevates spatial transcriptomics from discrete spot-based observations to continuous expression field modeling. By integrating conditional neural fields with histology-guided multiscale representation learning, the model achieves coherent reconstruction of spatial gene expression while maintaining consistency with the underlying measurement process.

The paper positions HiFi-ST as a principled approach for integrating morphological and molecular information in spatial omics. Its primary significance lies in its potential to improve spatial expression reconstruction accuracy and support downstream analyses, such as tumor microenvironment characterization and spatial immune structure identification. The authors note that while the current implementation is limited to 2D sections and relies on uniform sampling assumptions, the continuous neural field paradigm offers a generalizable direction for high-fidelity spatial transcriptomics inference.

Technical Summary of HiFi-ST: High-Fidelity Reconstruction of Continuous Spatial Transcriptomic Expression Fields via Conditional Neural Fields

More like this