Represent, Then Generate: Multimodal-Conditioned Time-Series Generation under Irregular Missingness
The paper introduces ReCoGen, a two-stage multimodal framework that decouples condition representation from generation to synthesize missing physiological signals from heterogeneous, irregularly missing clinical data, achieving superior downstream utility across multiple benchmarks.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to predict the weather, but you only have a broken thermometer that stops working whenever it rains, a wind gauge that only spins on Tuesdays, and a diary entry from your grandmother about how the air felt last summer. In the world of medical science, doctors face a similar puzzle every day. They rely on a constant stream of data—like heartbeats, blood sugar levels, and breathing rates—to keep patients safe. These are called "physiological time series." Think of them as the body's live broadcast, a non-stop movie of how a person is doing. However, this broadcast is often full of static, missing scenes, or gaps where the signal just disappears. Sometimes, the most important "camera" (like a sensor that measures blood pressure inside an artery) is too invasive or expensive to use on everyone, so doctors are left trying to guess what that missing movie looks like based on the other, imperfect cameras they do have.
This is where the field of "conditional generation" steps in. It's like a super-smart AI artist that tries to paint the missing parts of the movie using the clues from the other cameras. The goal is to create a fake signal that is so realistic and useful that a doctor could use it just like the real thing to spot danger early. But here's the catch: most of these AI artists are trained to work with only one type of clue at a time, or they get confused when the clues arrive at different times or have different kinds of missing pieces. They try to mash everything together into a messy pile, which often leads to a blurry, unhelpful painting.
Enter ReCoGen (Represent, then Generate), a new approach proposed by researchers Haochen Zhang and his team. They realized that before you can paint a masterpiece, you first need to understand the ingredients. Instead of throwing all the messy, broken data directly into the generator, ReCoGen uses a two-step "clean-up and create" strategy. First, it treats each type of signal (like heart rate or blood sugar) as its own special puzzle. It uses a "masked autoencoder"—think of this as a dedicated detective for each signal—who learns to fill in the missing gaps of that specific signal using only the context around it. This detective turns the messy, irregular data into a neat, compact list of "tokens" (like a summary note) that is robust and ready to use.
Once these detectives have done their job, the second stage begins. A powerful "flow-matching" generator takes these neat summary notes, along with static information about the patient (like their age or medical history), and synthesizes the missing target signal. The researchers tested this on three different medical datasets, including continuous glucose monitoring for diabetes and blood pressure monitoring in intensive care units. They found that ReCoGen was significantly better at creating useful signals than six other leading methods. In fact, on thirteen out of sixteen different test scenarios, the signals created by ReCoGen were just as good as, or even better than, the real signals when used to predict serious health events like sepsis or heart failure.
The paper suggests that the secret sauce isn't just in the generation part, but in how the conditions are represented. By separating the "understanding" phase from the "creating" phase, and by using a special attention mechanism to mix the different types of clues, ReCoGen manages to turn routine, imperfect signals into high-quality surrogates for the invasive or unavailable ones. While the authors note that this doesn't mean the fake signal is a perfect copy of reality (it might just be a cleaner version of the information the doctor already has), it proves that this two-stage method is a major step forward toward less invasive, lower-cost, and more continuous health monitoring.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.