Imputing relative abundances via Gaussian random fields

Human biology is rarely a single snapshot; it is a story unfolding over time. In fields like microbiome research, scientists track the tiny communities of bacteria living inside us, watching how they shift and change from week to week, month to month, or year to year. These communities are measured not by counting individual bacteria, but by looking at the relative proportions of different types, much like seeing how the ingredients in a soup change as the pot simmers. This approach is powerful because it reveals the dynamic health of a person's internal ecosystem. However, this kind of long-term tracking is notoriously difficult to keep perfect. People miss appointments, samples get lost in transit, or machines fail to read the data correctly. When a single measurement is missing from a long timeline, the story breaks. The continuous flow of data is interrupted, making it hard to see the true patterns of health or disease, and often forcing researchers to throw away valuable information just because a piece is missing.

To solve this problem, researchers Andrey Shternshis, Prashant Singh, and Luisa W. Hugerth have developed a new way to fill in those missing pieces of the biological puzzle. They created a mathematical framework that acts like a skilled storyteller, capable of imagining what a missing sample likely looked like based on everything else that was recorded. Instead of simply guessing the most obvious answer or copying a previous measurement, their method uses a sophisticated statistical approach called a Gaussian random field. This technique allows the model to understand the complex relationships between different bacteria within a single sample and how those relationships evolve over time for a specific person. It is designed to work even when the data is sparse, using a moderate number of parameters so it doesn't get confused by the noise, which is a common issue with more complex artificial intelligence models that require massive amounts of data to learn.

The core of this new method is its ability to handle "compositional" data, which means it respects the rule that all the parts of a sample must add up to a whole. If one type of bacteria appears to increase, the model understands that the relative amounts of others must decrease accordingly. Furthermore, it accounts for the fact that some bacteria might be present in such tiny numbers that they go undetected, treating them as very small values rather than zero, which keeps the mathematical logic sound. By generating not just one guess, but many possible versions of the missing data, the model captures the natural uncertainty of biology. This allows scientists to run their analyses on a whole set of completed datasets, ensuring that their final conclusions about health trends are robust and account for the gaps in the original records.

The researchers tested this framework on real-world data from three different studies involving human participants. In one study, they looked at gut bacteria from hundreds of pregnant women to see if they could predict who would develop postpartum depression. In another, they examined infants to forecast which children would develop a specific food allergy. In a third, they analyzed gut samples from middle-aged adults to see how well the model could reconstruct data for species that varied the most in their populations. The results were promising. When the model filled in the missing samples, it produced data that was remarkably close to the real measurements, outperforming older, simpler methods that just carried forward the last known value. More importantly, when they used these reconstructed datasets to predict health outcomes, the accuracy improved significantly. For the infants, the model helped correctly identify food allergies with a balanced accuracy of 0.72, a notable improvement over previous attempts. For the pregnant women, it helped forecast postpartum depression with a balanced accuracy of 0.64.

What makes this work particularly valuable is that it does not rely on black-box algorithms that are impossible to interpret. The model is transparent, meaning researchers can see exactly how it uses the available information—such as the time of the sample and the participant's specific characteristics—to make its predictions. It successfully bridges the gap between the messy reality of human studies, where missing data is the norm rather than the exception, and the need for complete, high-quality datasets to understand biological trends. By providing a way to reconstruct these broken timelines with statistical rigor, the framework offers a practical tool for scientists to recover lost insights and make more reliable predictions about human health, turning fragmented records into a coherent narrative of biological change.