🤖 machine learning

ConformalShift: Targeted Event Reordering Against Adaptive ECG Monitoring

The paper introduces ConformalShift, a novel attack that compromises adaptive ECG monitoring systems by strategically reordering authentic preceding events to suppress critical ventricular heartbeat classifications, demonstrating that the timing of information alone can undermine healthcare AI even when all underlying data and models remain unchanged.

Original authors: Arash Vashagh, Yasmin Vashagh

Published 2026-08-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Arash Vashagh, Yasmin Vashagh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a ship navigating through a foggy sea, relying on a high-tech radar to spot hidden icebergs. Usually, the radar just points at the water and says, "Iceberg ahead!" or "All clear." But sometimes, the radar is a bit unsure, so it gives you a list of possibilities: "It might be an iceberg, or maybe a whale, or just a cloud." To make this list smarter, the radar learns from the captain's past corrections. If the captain says, "Actually, that was a whale," the radar adjusts its settings for the next time it sees a whale. This learning happens after the fact, with a slight delay, like sending a message back to the radar station. The big question scientists are asking is: Does the order in which these messages arrive matter? If the messages get jumbled up—say, the "It was a whale" note arrives before the "It was a cloud" note—does the radar get confused and start missing real icebergs, even if the radar itself and the pictures of the water haven't changed at all? This is the world of "adaptive machine learning," where systems constantly tweak themselves based on new information, and it's a crucial field because we are starting to use these self-adjusting systems to keep people safe in hospitals, on the roads, and in our homes.

Now, meet the researchers who decided to play a trick on this kind of smart radar. They created a clever prank called ConformalShift. Instead of trying to trick the system by changing the pictures of the heart (the ECG waves), the labels, or the computer's brain, they simply shuffled the order in which the heartbeats were delivered to the monitor. Imagine a line of people waiting to get their tickets checked. The system learns from each person as they pass. The researchers found that if you quietly swap the positions of a few people in line—without changing who they are or what they look like—you can confuse the system's memory. Specifically, they showed that by rearranging the order of "authentic" heartbeats, they could trick a smart heart monitor into ignoring a dangerous, irregular heartbeat (a ventricular beat) that it would have normally caught.

Here is how the magic trick works. The monitor is designed to be extra careful. If a standard computer says, "This heartbeat looks normal," but the monitor's safety net is unsure, it keeps the "danger" option open just in case. This is called a "rescue." The researchers found that if they reordered the events leading up to a dangerous heartbeat, they could lower the monitor's safety threshold just enough to close that safety net. They did this by moving "safe" heartbeats that had already been checked earlier in the line, and pushing "missed" heartbeats later. It's like a teacher grading a stack of papers. If the teacher sees a bunch of easy A's first, they might get strict and grade the next paper harshly. But if they see a few tricky papers first, they might be more lenient. The researchers didn't change the papers; they just changed the order the teacher saw them in.

The results of this experiment were quite surprising. When they tested this on real heart data from the MIT–BIH database, the trick worked much better than just shuffling the papers randomly. For one type of computer model (called Extra Trees), the attack successfully hid the dangerous heartbeat 66.7% of the time, whereas random shuffling only worked 4.4% of the time. For another model (HistGradientBoosting), it worked 60.0% of the time compared to 12.0% for random shuffling. Even when they tried this trick on a completely different set of heart data (from the INCART database) without retraining the system, it still worked about 33.3% of the time, which was far better than the random chance of about 5%.

The paper also tested how strict the rules were. They found that if they limited how far they could move the people in line (a "displacement budget"), the trick became weaker. When they reduced the allowed shuffling distance, the success rate dropped significantly, showing that the attack relies on having enough freedom to rearrange the order.

The most important takeaway isn't that the heart monitors are broken, but that they are vulnerable to a very specific kind of confusion: timing. The researchers proved that you don't need to hack the sensors or the computer code to fool an adaptive system; you just need to mess with the schedule. They explicitly ruled out the idea that you need to change the heart signals or the labels to cause this problem. The paper suggests that this is a real, measurable weakness in how these systems learn from delayed feedback. It's a reminder that in the future, when we build smart medical devices, we need to make sure they don't just look at what information they get, but also when and in what order it arrives. If we don't protect the order of events, even the smartest, most honest system can be tricked into missing a warning sign.

Technical Summary: ConformalShift

Problem Statement
Adaptive conformal prediction is increasingly used in healthcare, specifically for continuous electrocardiogram (ECG) monitoring, to recover clinically significant heartbeat classes (such as ventricular ectopic beats) that point classifiers may miss. These systems update their coverage parameters based on observed errors in a delayed feedback loop, typically processing events in a first-in, first-out (FIFO) order. While existing adversarial research focuses on modifying input signals, labels, or model parameters, this paper identifies a novel vulnerability: the decision-making process of adaptive monitors is sensitive to the order in which authentic events are processed. The authors posit that an adversary could manipulate the transmission timing of verified, unmodified events to alter the monitor's internal state, thereby compromising the system's ability to "rescue" missed critical events without changing any physiological data or classifier outputs.

Methodology
The paper introduces ConformalShift, a bounded event-reordering attack designed to suppress the inclusion of a target ventricular class in the conformal prediction set. The attack operates under the following constraints and mechanisms:

Threat Model: The adversary cannot modify ECG waveforms, verified labels, classifier scores, or the event multiset. The target event itself remains fixed. The attacker can only reorder the $W$ authentic events immediately preceding the target within a bounded displacement budget $d$ .
Mechanism: The adaptive monitor maintains class-specific thresholds that evolve based on "miss" and "covered" feedback. A "miss" (where the nonconformity score exceeds the threshold) increases the threshold for that class, while a "covered" event decreases it. ConformalShift exploits the delayed feedback mechanism ( $\delta$ ) by reordering preceding events to ensure that "covered" ventricular feedback matures (updates the threshold) before the target event is evaluated, while pushing "missed" ventricular feedback to mature after.
Search Strategy: Since the space of permutations is factorial, the authors employ a constrained search method. This involves a displacement-aware beam search followed by local refinement (adjacent swaps, long-range swaps, and relocations) to find a feasible schedule $\pi$ that maximizes the suppression margin $m(\pi) = a_{t^*,v} - q^{\pi}_{v,t^*}$ , where $a$ is the nonconformity score and $q$ is the adaptive threshold.
Objective: The attack succeeds if the reordering causes the ventricular threshold to drop below the target's fixed nonconformity score, excluding the ventricular class from the prediction set, whereas the original FIFO order would have included it.

Key Contributions

Formulation of Event Reordering as an Attack: The authors formalize bounded event reordering as a method to manipulate the threshold state of an adaptive conformal monitor, distinct from traditional input perturbation attacks.
Constrained Search Algorithm: They developed a specific search method (beam search with local refinement) to identify feasible event permutations that are likely to suppress a target class under displacement constraints.
Empirical Evaluation: The study evaluates ConformalShift across two ECG datasets (MIT–BIH and INCART) and two frozen classifier families (Extra Trees and HistGradientBoosting), comparing the attack against random feasible schedules and analyzing the impact of the displacement budget.

Results
The experiments demonstrate that ConformalShift significantly outperforms random scheduling in suppressing target events:

MIT–BIH (Disjoint Confirmation Records): The attack suppressed 66.7% of eligible targets for Extra Trees and 60.0% for HistGradientBoosting. In contrast, random feasible schedules suppressed only 4.4% and 12.0% of targets, respectively.
MIT–BIH (Full DS2): On the full descriptive set, suppression rates were 63.0% (Extra Trees) and 49.3% (HistGradientBoosting), compared to random baselines of 7.7% and 10.8%.
Cross-Dataset Transfer: Configurations trained on MIT–BIH transferred to the INCART dataset without specific tuning, achieving a 33.3% suppression rate for both models, significantly higher than the random baselines (5.0% and 4.5%).
Displacement Budget Ablation: Reducing the maximum displacement budget ( $d$ ) weakened the attack. For example, on MIT–BIH with Extra Trees, reducing $d$ from 8 to 4 dropped the success rate from 66.7% to 16.7%, confirming that the attack relies on the ability to shift feedback timing.

Significance and Claims
The paper claims that adaptive monitors in healthcare are vulnerable to temporal integrity attacks. The results demonstrate that even when waveforms, labels, classifier outputs, and event contents remain unchanged, the timing of authentic information processing can compromise the system's safety. Specifically, the large gap between targeted reordering and random scheduling indicates that the vulnerability is structured and exploitable, not merely a stochastic consequence of reordering.

The authors conclude that robustness evaluations for adaptive healthcare monitoring must extend beyond input and model attacks to include feedback order, delay, and temporal integrity. They suggest that practical defenses may require authenticating timestamps, rejecting stale feedback, or constraining out-of-order delivery to preserve the safety benefits of adaptive conformal prediction. The study is limited to white-box attackers, retrospective datasets, and specific update rules, suggesting a need for future work on partial-knowledge attacks and other adaptive methods.

Technical Summary: ConformalShift

More like this