🤖 AI

Explainable Fall Detection for Elderly Care via Temporally Stable SHAP in Skeleton-Based Human Activity Recognition

This paper proposes a lightweight, real-time skeleton-based fall detection framework that integrates an efficient LSTM model with a temporally stable SHAP aggregation strategy (T-SHAP) to generate reliable, clinically trustworthy explanations by smoothing feature attributions over time, achieving 94.3% accuracy and enhanced explanation faithfulness on the NTU RGB+D dataset.

Original authors: Mohammad Saleh, Azadeh Tabatabaei

Published 2026-04-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Mohammad Saleh, Azadeh Tabatabaei

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are building a smart security guard for an elderly care facility. This guard's job is to watch over residents and instantly shout, "Someone has fallen!" if they see a tumble.

For a long time, computer scientists have been building these guards using powerful AI. But there's a big problem: The AI is a "Black Box."

It can tell you that someone fell with 94% accuracy, but it can't tell you why. It's like a doctor saying, "You're sick," but refusing to show you the X-ray or explain which symptom caused the diagnosis. In a hospital or a nursing home, doctors and caregivers need to trust the system. They need to know: Did the AI see the leg buckle? Did it see the spine twist? Or did it just get lucky?

This paper introduces a new system that solves two problems at once: it's fast enough to be real-time and clear enough to be trusted.

Here is the breakdown of how they did it, using some simple analogies.

1. The Problem: The "Flickering Lightbulb"

The researchers looked at how existing AI explains its decisions. They found that when the AI looks at a video of a person falling, it tries to explain itself frame-by-frame (like looking at a flipbook one page at a time).

The Issue: Because the AI looks at each split-second independently, its explanation "flickers."
- Frame 1: "The left knee is the problem!"
- Frame 2: "No, it's the right elbow!"
- Frame 3: "Actually, it's the nose!"
The Result: To a human doctor, this looks like noise. It's like trying to read a book where the letters jump around on the page every time you blink. You can't make a decision based on that.

2. The Solution: The "Smoothie" (T-SHAP)

To fix this, the authors created a new method called T-SHAP (Temporally Stable SHAP).

Think of the AI's explanation like a smoothie.

Standard AI (SHAP): If you just throw random fruits into a blender for one second, you get a chunky, uneven mix. Some bites are all apple, some are all banana.
T-SHAP: This method takes those individual frames and blends them together over a short window of time (like a 5-second clip). It smooths out the "chunky" noise.
The Analogy: Instead of seeing a flickering lightbulb, T-SHAP turns it into a steady, glowing lamp. It tells the doctor: "Over the last few seconds, the person's knees and spine were unstable, leading to the fall." This is a stable, reliable story that a human can trust.

3. The Engine: The "Lightweight Sports Car"

Many modern AI models are like heavy, fuel-guzzling trucks. They are very smart (high accuracy), but they are slow and require massive computer power to run. You can't easily put a truck in a small apartment or a portable medical device.

The authors chose a Lightweight LSTM model.

The Analogy: Think of this as a nimble sports car. It's not the biggest or the most complex vehicle in the world, but it is incredibly fast and efficient.
The Result: The system can analyze a fall and give an explanation in less than 25 milliseconds. That is faster than the blink of an eye. This means it can run on standard, affordable hardware in a real nursing home without needing a supercomputer.

4. The Proof: Does it actually work?

The researchers tested their system on a dataset of people falling (the NTU RGB+D dataset). They compared their "Smoothed Sports Car" against other methods:

Accuracy: It caught falls correctly 94.3% of the time. (Very high!)
Trustworthiness: They used a test called "Faithfulness." Imagine you take the most important clues the AI found and hide them. If the AI is good, it should get confused and stop predicting "Fall."
- The old methods got confused only a little bit.
- T-SHAP got confused a lot when its clues were hidden. This proves that T-SHAP is actually looking at the real reasons for the fall, not just guessing.
The "Doctor's Eye": When they looked at the heatmaps (the visual explanations), T-SHAP correctly highlighted the knees, hips, and spine—the exact body parts that biomechanics experts know are involved in a fall. It didn't just point at random pixels; it pointed at the biomechanics.

Why This Matters

In the real world, if an AI says "Fall Detected," a nurse needs to know why so they can act.

If the AI says, "The person's knee buckled," the nurse knows to check for leg weakness.
If the AI says, "The person lost balance," the nurse knows to check for dizziness.

By making the AI's "thought process" steady, fast, and easy to understand, this paper bridges the gap between complex math and human care. It turns a scary "Black Box" into a transparent, helpful assistant that doctors and caregivers can actually trust with people's lives.

In short: They built a fast, smart AI that doesn't just say "Fall!" but also calmly explains, "Here is exactly how the fall happened," without stuttering or flickering.

1. Problem Statement

Fall detection in elderly care is a safety-critical application requiring not only high classification accuracy but also reliable, interpretable explanations that clinicians can trust. While deep learning models (e.g., GCNs, Transformers) achieve high accuracy in skeleton-based Human Activity Recognition (HAR), they often suffer from:

Lack of Transparency: They function as "black boxes," making it difficult to validate their decisions in clinical settings.
Temporal Instability in Explanations: Existing post-hoc explainability methods (like standard SHAP or Grad-CAM) applied frame-by-frame to sequential data produce noisy, high-variance attribution maps. These fluctuate rapidly between consecutive frames, obscuring coherent motion patterns and making it difficult for clinicians to identify the specific biomechanical causes of a fall (e.g., loss of balance vs. impact).
Computational Cost: State-of-the-art models are often too computationally heavy for real-time deployment on mid-range hardware.

2. Methodology

The authors propose a lightweight, end-to-end framework that balances accuracy, efficiency, and temporal stability in explanations.

A. Model Architecture

Backbone: A Lightweight LSTM (Long Short-Term Memory) network.
- Input: 3D skeleton coordinates (25 joints × 3 dimensions = 75 features per frame).
- Structure: Single-layer LSTM (128 hidden units) followed by a Dense layer and Softmax.
- Rationale: LSTMs offer linear computational complexity $O(T)$ , making them suitable for real-time inference (<25ms), unlike quadratic-complexity Transformers or heavy Graph Convolutional Networks (GCNs).
Dataset: The NTU RGB+D Dataset, specifically focusing on fall-related classes (falling, sitting, standing, picking up) for binary classification.

B. The Core Innovation: T-SHAP (Temporally Stable SHAP)

To address the temporal instability of standard SHAP, the authors introduce T-SHAP, a post-hoc aggregation strategy.

Mechanism: Instead of treating each frame independently, T-SHAP applies a linear smoothing operator (moving average) over a temporal window $W_t$ $W_{t}$ to the SHAP attribution values.
- Formula: $\tilde{\phi}_{i,t} = \frac{1}{|W_t|} \sum_{k \in W_t} \phi_{i,k}$
- Where $\phi$ represents the original SHAP value for feature $i$ at time $t$ , and $W_t$ is a window of size $2w+1$ .
Theoretical Guarantees: T-SHAP preserves the theoretical properties of Shapley values (local accuracy, consistency, missingness) because the aggregation is a linear transformation applied after the calculation.
Effect: It acts as a low-pass filter, reducing high-frequency noise (jitter) while preserving semantically meaningful, sustained motion patterns.

C. Evaluation Metrics

Faithfulness: Measured using Area Under the Perturbation Curve (AUP). This metric quantifies how much the model's prediction confidence drops when the top-ranked features (according to the explanation) are removed. Higher AUP indicates the explanation is more causally faithful to the model's decision.
Temporal Stability: Measured by the variance of attribution scores across consecutive frames.
Latency: End-to-end inference time.

3. Key Contributions

T-SHAP Framework: Introduction of a temporally aware, post-hoc aggregation strategy that stabilizes SHAP attributions for sequential data without requiring additional model training.
Lightweight Real-Time System: Demonstration that a simple LSTM architecture can achieve competitive accuracy (94.3%) with significantly lower computational overhead than GCNs or Transformers, suitable for edge/clinical devices.
Quantitative Validation of XAI: Moving beyond qualitative visualization, the paper provides rigorous quantitative evidence (via AUP and variance metrics) that T-SHAP produces more faithful and stable explanations than standard SHAP and Grad-CAM.
Biomechanical Alignment: The explanations generated by T-SHAP align with established clinical observations of fall dynamics (e.g., highlighting lower-limb instability and spinal alignment changes), validating the model's decision-making process.

4. Experimental Results

Classification Performance:
- Accuracy: 94.3%
- Precision/Recall/F1: ~94%
- Latency: Total inference time is <25 ms (4.8 ms for LSTM prediction + 12–20 ms for T-SHAP explanation), satisfying real-time constraints (<100 ms).
Explainability Performance (Faithfulness - AUP):
- T-SHAP: 0.91 (Highest)
- Standard SHAP: 0.89
- Grad-CAM: 0.82
- Interpretation: T-SHAP identifies features that are more causally linked to the prediction, as removing them causes a sharper drop in model confidence.
Temporal Stability:
- T-SHAP reduced temporal variance by 24% compared to standard SHAP.
- Visualizations showed T-SHAP produces coherent, continuous heatmaps highlighting specific joints (spine, knees, hands) over time, whereas standard SHAP showed fragmented, noisy activations.
Sensitivity Analysis: A window size of $w=2$ (5-frame window) was found to be optimal, balancing noise reduction with the preservation of fine-grained temporal details.

5. Significance and Impact

Clinical Trust: By providing stable, biomechanically consistent explanations, the framework bridges the gap between AI predictions and clinical reasoning. Clinicians can verify why a fall was detected (e.g., "progressive loss of balance in the lower limbs") rather than just receiving a binary alert.
Real-World Deployment: The combination of high accuracy and low latency makes this system viable for deployment in assisted living facilities and home monitoring systems on mid-range hardware.
Paradigm Shift: The paper argues that in safety-critical domains, optimizing the stability and reliability of explanations is as important as maximizing raw accuracy. It demonstrates that complex architectures are not always necessary if the interpretability and temporal coherence of the model are rigorously addressed.
Future Directions: The T-SHAP approach is model-agnostic and can be extended to other sequential domains (rehabilitation, sports biomechanics) and more complex architectures (Transformers, GCNs) to solve the "temporal instability" problem in broader XAI applications.