← Latest papers
💻 bioinformatics

TS-Zarr: a cloud-optimized Zarr layout for viewing and analyzing biomedical time series

TS-Zarr is a cloud-optimized Zarr v3 layout for biomedical time series that stores precomputed summary pyramids alongside raw data, enabling scalable, low-latency visual analysis and targeted parallel reads without requiring local downloads or dedicated resampling servers.

Original authors: Shah, R., Garai, S., De Freitas, D. D., Truong, V. Q., Ganglberger, W., Paron, T., Westover, M. B., Sinha, N., Wagenaar, J.

Published 2026-11-18
📖 5 min read🧠 Deep dive

Original authors: Shah, R., Garai, S., De Freitas, D. D., Truong, V. Q., Ganglberger, W., Paron, T., Westover, M. B., Sinha, N., Wagenaar, J.

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Medical researchers and doctors often rely on time series data to understand the human body. Imagine a recording that tracks electrical signals from the brain, heart, or muscles over many hours or even days. These recordings capture hundreds of different channels simultaneously, creating files so massive they can fill entire hard drives. To make sense of this data, scientists need to look at it visually, much like reading a long line of text. However, when a recording spans a full day, trying to display every single data point on a computer screen is impossible; the screen simply does not have enough space. If a researcher wants to see a ten-second window, the computer must squeeze thousands of data points into a few pixels. If they want to see a whole day, it must squeeze millions of points into the same space. Without a special method to reduce this data, the computer either freezes trying to process it, or the image becomes a solid, unreadable block of color that hides important details.

For years, the solution to this problem has been to use a powerful server to do the heavy lifting. When a user asks to see a specific part of a recording, the server reads the raw data, calculates a simplified version that fits the screen, and sends that image back. This works well for a few people, but it breaks down when many users try to look at the same data at once, or when the time window they want to see is very long. The server gets overwhelmed, and the wait times become frustratingly long. Furthermore, if a researcher wants to analyze the data on their own computer, they often have to download the entire multi-gigabyte file first, which is slow and inefficient. The challenge has been to find a way to let anyone view these massive recordings instantly, without needing a supercomputer or a massive download.

A team of researchers has developed a new way to store and view this kind of data called TS-Zarr. Instead of keeping the data in a single, monolithic file, they have reorganized it into a structure designed specifically for cloud storage. In this new system, the raw data is still there, but it is accompanied by a set of pre-calculated summaries at different levels of detail. Think of these summaries as a series of maps: one map shows the entire country, another shows the state, another the city, and so on, down to the street level. When a researcher wants to see a wide view of a twenty-four-hour recording, the computer does not need to read the millions of individual data points. Instead, it simply grabs the "country map" summary, which is tiny and loads instantly. If the researcher zooms in to look at a ten-second window, the computer switches to a more detailed "city map" summary. If they zoom in even further to see a fraction of a second, it finally reads the raw data.

The researchers found that this approach changes everything about how fast and how many people can view the data. In their tests, they compared this new method against older file formats and the traditional server-based approach. When a single user looked at a ten-second window, the new system was already faster. But the real difference appeared when the window grew to a full day. The traditional server, which had to read and crunch the entire day's worth of raw data for every single request, took forty-five seconds to respond. In contrast, the new system, which only needed to fetch a tiny pre-calculated summary, responded in less than half a second. The difference became even more dramatic when many people tried to view the data at the same time. The server quickly became overloaded and failed, while the new system handled hundreds of users simultaneously without slowing down, because each user was fetching small, independent pieces of data directly from the cloud.

This new layout also solves the problem of downloading massive files. Because the system is built to let computers fetch only the specific channels and time windows they need, a researcher can look at a single channel from a specific hour without ever touching the rest of the recording. The data is stored in a way that allows many computers to grab these small pieces at the same time, making the process incredibly efficient. The researchers also showed that the summaries are smart enough to preserve the most important features of the signal. They do not just average the data, which would smooth out important spikes and dips; instead, they keep track of the highest and lowest points in every section. This ensures that when a doctor or scientist looks at the screen, they see the true range of the signal, even when it is compressed to fit a small space.

The team demonstrated that this method works for recordings that are twenty-four hours long with hundreds of channels, a scale that previously required massive storage and slow servers. By moving the work of simplifying the data from a central server to the storage itself, they have made it possible for researchers to explore years of medical data as easily as they would a single day. The system does not require users to download local copies of terabyte-sized files, nor does it rely on a dedicated server that might crash under pressure. It simply allows the data to be viewed instantly, no matter how wide the window or how many people are looking, turning what was once a bottleneck into a seamless experience. This shift means that the focus can return to the science itself, rather than waiting for the computer to catch up.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →