📊 statistics

Event-Aligned Analysis of Multi-Rater Pain Assessments Using Continuous Wearable Physiology

This paper introduces a rater-aware, event-aligned framework that maps continuous wearable physiological data to discrete pain-change events from specific raters, revealing significant inter-rater disagreement and suggesting that pain-physiology relationships are rater-dependent rather than invariant.

Original authors: Saba A. Farahani, Elahe Khatibi, Thomas D. Hughes, Ariana M. Nelson, Hung Cao, Amir M. Rahmani

Published 2026-06-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Saba A. Farahani, Elahe Khatibi, Thomas D. Hughes, Ariana M. Nelson, Hung Cao, Amir M. Rahmani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand why a car engine is making a strange noise. You have three different people looking at the car: the driver (the patient), a mechanic in the garage (the clinician), and a passenger who knows a bit about cars (the nurse).

In most previous studies, researchers would ask these three people, "How loud is the noise?" and then take their answers, average them out, and say, "Okay, the noise is a 5 out of 10." They would then look at the car's sensors (like the temperature gauge or the RPM meter) to see if those numbers matched that "5."

This paper says: "Wait a minute. That's not how it works."

Here is the simple breakdown of what the researchers actually did and found:

1. The Problem: Everyone Sees a Different "Noise"

The researchers realized that the driver, the mechanic, and the passenger often disagree on when the noise gets louder and how much louder it gets.

The driver might say, "It just got loud!"
The mechanic might say, "No, it's been loud for a while."
The passenger might say, "I didn't notice a change."

Previous computer models treated these disagreements as "mistakes" or "noise" to be ignored. This study argues that these disagreements are actually important clues. The driver, the nurse, and the clinician are looking at the car through different lenses, and they might be reacting to different things.

2. The New Tool: The "Event" Camera

Instead of trying to force everyone to agree on a single number, the researchers built a new way of looking at the data. They called it a "Rater-Aware, Event-Aligned Framework."

Think of it like this:

Old Way: Taking a blurry photo of the whole day and trying to guess what happened.
New Way: Setting up a motion-activated camera. Every time anyone says, "Hey, the pain just changed!" the camera snaps a picture.

They didn't ask, "What is the pain level?" Instead, they asked, "When did the pain level change?"

If the patient says the pain went up, the camera snaps a photo of the body's sensors (heart rate, skin sweat, temperature) right before that moment.
If the nurse says the pain went up, the camera snaps a different photo of the sensors right before that moment.

They kept the "who" (the rater) attached to every single photo. They didn't mix the patient's photos with the nurse's photos.

3. The Discovery: Different Eyes, Different Signals

When they looked at the photos (the data) they found some surprising things:

They Disagree a Lot: Even when they were looking at the same patient at roughly the same time, the patient, nurse, and clinician often reported pain changes at completely different times. It wasn't just a matter of one person being "wrong"; they were noticing different things.
The Body Reacts Differently Depending on Who Notices:
- When the Clinician said pain went up: The body's sensors (like skin sweat and heart rate) showed a strong, clear "warning sign" just before the clinician spoke. It was like the car engine revving up loudly before the mechanic noticed the noise.
- When the Patient said pain went up: The body's sensors were much quieter or behaved differently. The heart rate actually seemed to drop slightly before the patient spoke, rather than spike.
- When the Nurse said pain went up: The signals were somewhere in the middle.

4. The Big Takeaway

The paper concludes that pain and the body's signals are not a one-size-fits-all relationship.

If you try to build a computer program to predict pain by averaging everyone's opinions, you might be missing the real story. The body might be screaming "I'm in pain" in a way that the clinician sees clearly, but the patient doesn't feel yet (or vice versa).

By keeping the "who" in the equation and looking at the exact moment a change happens, the researchers found that the body's reaction depends entirely on who is reporting the pain.

In short: You can't just mash all the opinions together and expect to understand the body's signals. You have to listen to the driver, the mechanic, and the passenger separately to understand what the car is actually doing.

Technical Summary: Event-Aligned Analysis of Multi-Rater Pain Assessments Using Continuous Wearable Physiology

Problem Statement

Pain assessment in clinical settings is inherently subjective and varies significantly across raters (patients, nurses, and clinicians) due to differing information sources and interpretation criteria. However, current computational approaches to pain monitoring typically assume a single "ground-truth" pain label, often prioritizing patient self-report or aggregating ratings from multiple observers. This assumption obscures meaningful differences in how pain is perceived and reported. Furthermore, a structural mismatch exists between clinical data: pain ratings are sparse, irregular, and rater-dependent, whereas wearable physiological signals (e.g., EDA, HR) are dense and continuous. Existing methods often address this via fixed windowing or label interpolation, treating pain scores as static labels rather than dynamic events, thereby failing to capture the temporal dynamics preceding pain changes or the specific perspectives of different raters.

Methodology

The authors propose a rater-aware, event-aligned analysis framework designed to preserve rater identity and model pain as a sequence of discrete change events rather than static labels. The methodology consists of the following steps:

Dataset: The study utilized a clinically collected dataset from 24 participants undergoing spine-related pain procedures. Data included synchronized pain assessments from patients, nurses, and clinicians (using a 0–10 Numerical Rating Scale) and continuous physiological signals (Electrodermal Activity [EDA], Heart Rate [HR], and Skin Temperature) recorded via wrist-worn Empatica E4 and EmbracePlus devices.
Event Definition: Instead of using raw scores, the framework defines pain-change events. For each rater independently, an event is triggered at time $t$ when the absolute difference between consecutive pain ratings exceeds a threshold ( $\delta = 1$ , the smallest observable change on the NRS). The study focuses specifically on pain-increase events.
Window Extraction: For each identified event, a fixed-length physiological window is extracted: 15 seconds pre-event ( $T_{pre}$ ) and 30 seconds post-event ( $T_{post}$ ). This duration was selected to capture short-term autonomic responses (known to have 1–15s latencies) without encroaching on neighboring events.
Signal Processing: Physiological signals were standardized within each subject to reduce inter-individual baseline differences. A pre-event slope feature was computed as the least-squares linear trend of the standardized signal over the $[-15, 0]$ second interval.
Analysis:
- Inter-rater Disagreement: Quantified by calculating the absolute difference in reported pain changes between rater pairs within 10-minute windows.
- Rater-Specific Dynamics: Event-aligned physiological responses were aggregated by rater group. Standardized linear models were fitted separately for each rater to evaluate the relative importance of physiological features (EDA, HR, Temp) in association with reported pain changes. These models were used for interpretability and comparison, not predictive evaluation.

Key Contributions

The paper makes three primary contributions:

Framework Development: A rater-aware, event-aligned framework that converts sparse, rater-specific pain ratings into discrete pain-change events and aligns continuous wearable signals to these events while preserving rater identity.
Characterization of Disagreement: A systematic characterization of inter-rater disagreement in pain-change reporting using clinically collected wearable data, demonstrating that disagreements are not merely noise but reflect distinct assessment perspectives.
Exploratory Evidence: Preliminary evidence suggesting that physiological dynamics preceding reported pain increases differ across patients, nurses, and clinicians, indicating that pain–physiology relationships may not be rater-invariant.

Results

Inter-Rater Disagreement: Substantial disagreement was observed across all rater pairs, particularly between patients and clinicians. The median number of events per subject was comparable across raters, indicating that disagreement stemmed from when and how changes were identified rather than reporting frequency.
Physiological Divergence: Pre-event physiological trends varied significantly by rater:
- EDA: Clinician-reported pain increases were typically preceded by a gradual, stronger rise in EDA compared to patient- or nurse-reported events, which showed smaller or more variable changes.
- Heart Rate (HR): Patient-reported pain changes were preceded by a gradual decrease in HR, whereas clinician-reported events showed an increasing HR trend approaching the event. Nurse-reported events fell between these patterns.
- Skin Temperature: Showed weaker and less consistent associations across raters, suggesting limited discriminative value in short pre-event windows.
Feature Importance: Standardized linear models indicated that EDA had stronger associations with pain changes reported by nurses and clinicians than by patients. HR contributed more prominently to clinician-reported pain changes.

Significance and Claims

The paper claims that treating pain assessments as interchangeable labels or aggregating them into a single ground truth may mask meaningful physiological patterns. The findings suggest that the relationship between wearable physiology and pain is rater-dependent.

Clinical Implications: The authors argue that a rater-aware, event-centered perspective is necessary for interpreting wearable data in real-world clinical settings. They propose that future systems should model multiple rater perspectives or explicitly incorporate rater-specific decision criteria, rather than enforcing agreement through label aggregation. This could enable clinical decision support tools that flag discrepancies between patient self-reports and clinician observations in real time.
Modesty of Claims: The authors explicitly characterize their findings as exploratory. Due to the modest cohort size ( $n=24$ ), limited event counts (41–47 per rater), and the repeated-measures structure of the data, the results are not presented as definitive statistical proof. They acknowledge potential confounds (e.g., procedural anticipation, movement, medication timing) that may influence the observed physiological patterns. The study calls for larger studies with formal statistical testing (e.g., mixed-effects models) to confirm these patterns.

In conclusion, the paper posits that preserving rater identity and temporal structure through event-aligned analysis provides a principled foundation for integrating wearable physiology with clinical pain assessments, moving beyond the limitations of single-label assumptions.