💻 computer science

Edge-Efficient Two-Stream Multimodal Architecture for Non-Intrusive Bathroom Fall Detection

This paper proposes an edge-efficient two-stream multimodal architecture that synergistically combines a Motion-Mamba branch for mmWave radar signals and an Impact-Griffin branch for floor vibrations to achieve high-accuracy, low-latency, and energy-efficient non-intrusive fall detection in bathrooms while effectively suppressing object-drop confounders.

Original authors: Haitian Wang, Yiren Wang, Xinyu Wang, Sheldon Fung, Atif Mansoor

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haitian Wang, Yiren Wang, Xinyu Wang, Sheldon Fung, Atif Mansoor

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine an elderly person living alone in a bathroom. It's a dangerous place: the floor is slippery, the walls are hard, and if they fall, no one might hear them. Cameras are too invasive (privacy issues), and wearable devices are often forgotten or break when wet.

The researchers from the University of Western Australia have built a "Digital Bodyguard" that solves this problem without cameras or watches. They created a smart system that uses two different "senses" to know exactly when someone has fallen, even while the shower is running.

Here is how their system works, explained with simple analogies:

1. The Two Senses: The "Eagle Eye" and the "Floor Ear"

Instead of relying on just one sensor, this system uses two distinct "streams" of information, like a detective using two different clues:

The Motion Stream (The Eagle Eye): This uses a mmWave radar (like a high-tech bat sonar). It watches the air. It can see a person's body moving, slowing down, or collapsing before they hit the ground. It's great at spotting the "falling" action.
- The Problem: If you drop a heavy soap bottle, the radar sees an object falling and might get confused, thinking it's a person.
The Impact Stream (The Floor Ear): This uses a vibration sensor attached to the floor. It listens for the "thud" when something hits the ground.
- The Problem: If a heavy towel rack falls, the floor shakes just like a person falling. The vibration sensor alone can't tell the difference between a person and a heavy object.

2. The Brain: The "Smart Matchmaker"

The magic of this paper isn't just having two sensors; it's how they talk to each other. The researchers built a special AI brain that acts like a matchmaker.

The Old Way: Previous systems would just look at the radar, then look at the vibration, and guess. It was like trying to solve a puzzle by looking at two pictures separately.
The New Way (Two-Stream Architecture): Their system checks the timing perfectly. It asks: "Did the radar see a body collapse exactly when the floor shook?"
- Scenario A (A Fall): Radar sees a body drop + Floor shakes hard = ALARM!
- Scenario B (Dropped Soap): Radar sees an object drop + Floor shakes a little = False Alarm, ignore it.
- Scenario C (Dropped Towel Rack): Radar sees nothing (no body moving) + Floor shakes hard = False Alarm, ignore it.

They call this "Cross-Conditioned Fusion." Think of it as a bouncer at a club who only lets you in if you have both a ticket (radar signal) and an ID (vibration signal). If you have one but not the other, you don't get in (no alarm).

3. The "Lightweight" Engine

Usually, smart AI brains are heavy and need powerful computers. But this system is designed to run on a Raspberry Pi 4B, which is a tiny, cheap computer the size of a credit card.

The Analogy: Imagine a Formula 1 race car engine. Usually, they are huge and thirsty. This team built a tiny, efficient engine that runs on a bicycle but still wins the race.
Why it matters: Because it's so efficient, it can run 24/7 in a bathroom without draining batteries or needing a massive server room. It reacts in 15.8 milliseconds (faster than a human blink), ensuring help arrives immediately.

4. The Training Ground: The "Wet Bathroom Gym"

To teach this AI, the researchers didn't just use dry data. They built a real bathroom mock-up and ran the shower while people (and objects) fell.

They recorded over 3 hours of data.
They included tricky situations: walking while wet, squatting, dropping heavy mops, and dropping soap.
This ensured the AI learned to ignore "noise" (like water splashing or heavy objects) and only scream when a human fell.

The Results: A Superhero Performance

When they tested their "Digital Bodyguard" against other systems:

Accuracy: It got it right 96.1% of the time.
Speed: It was more than twice as fast as the next best system.
Energy: It used less electricity (saving about 25% energy per check).
Safety: It caught 88% of actual falls (a very high number), meaning very few people were missed.

Summary

This paper presents a privacy-friendly, super-fast, and energy-efficient safety net for the elderly. By combining a "radar eye" and a "vibration ear" with a smart AI that knows how to tell the difference between a falling person and a falling soap bottle, they have created a system that could save lives in the most dangerous room of the house, all while running on a tiny, affordable computer.

1. Problem Statement

Falls in wet bathroom environments pose a critical safety risk for seniors living alone. Existing solutions face significant limitations:

Privacy & Intrusiveness: Camera and microphone systems are unsuitable for bathrooms due to privacy concerns; wearables are often neglected or fail in wet conditions.
Sensor Limitations:
- mmWave Radar: Effective at capturing pre-impact motion but prone to false alarms from dropped objects, specular reflections, and multipath clutter in confined, wet spaces.
- Floor Vibration: Excellent for detecting impact energy but lacks context (cannot distinguish human falls from heavy object drops) and is sensitive to floor construction.
Current Fusion Flaws: Existing multimodal approaches often rely on simple early feature concatenation or decision-level score averaging. These methods treat sensors as independent classifiers, failing to explicitly model the causal link between motion (collapse) and impact, and often ignore timing drift, sampling mismatches, and the strict latency/energy constraints of low-power edge devices.

2. Methodology

The authors propose an end-to-end, alignment-aware two-stream architecture designed for deployment on low-power edge hardware (Raspberry Pi 4B).

A. Data Preprocessing & Alignment

Synchronization: Raw triaxial vibration and mmWave radar signals are resampled to a common rate (100 Hz).
Time Alignment: A Generalized Cross-Correlation with Phase Transform (GCC-PHAT) estimates inter-sensor delay ( $\Delta\tau$ ) to correct for clock drift and transmission latency, avoiding heavy dynamic time warping.
Feature Extraction:
- Radar: Converted to kinematic descriptors (range, velocity, energy) and background-subtracted.
- Vibration: Denoised via wavelet shrinkage and converted to a scalar energy envelope.
Windowing: Data is sliced into 2.56s windows with 50% overlap, ensuring the impact peak aligns with the model's receptive field.

B. Two-Stream Architecture

The model processes modalities through specialized branches before fusing them:

Motion–Mamba Branch (Radar):
- Goal: Capture long-range motion patterns (pre-impact trajectory and collapse) while suppressing clutter.
- Mechanism: Uses LSK1D (Large Selective Kernel 1D) blocks for local context and Mamba2Block1D (State-Space Models) for efficient, linear-time long-range temporal modeling.
- Tokenization: A Switch–MoE (Mixture of Experts) adapter condenses the sequence into a motion token, allowing experts to specialize in different motion regimes (e.g., walking vs. falling).
Impact–Griffin Branch (Vibration):
- Goal: Detect sharp impact transients and cross-axis coupling while ignoring stationary structural tones.
- Mechanism: Uses LSK1D for multi-scale feature extraction and a Griffin block (Gated Linear Recurrent Unit) to update long-term memory. This allows rapid response to impacts while maintaining stability for low-amplitude noise.
- Adaptation: Applies inter-channel attention based on robust covariance to handle variations in sensor mounting and floor stiffness.

C. Cross-Modal Fusion

Instead of simple concatenation, the fusion module explicitly models the causal relationship:

Cross-Conditioning: Radar features are reinforced only when supported by nearby vibration activity (and vice versa) using local attention windows. This suppresses "vibration-only" shocks (object drops) and "radar-only" clutter.
Low-Rank Bilinear Interaction: The motion and impact tokens interact via a low-rank bilinear layer, preserving multiplicative structure without the parameter overhead of a full bilinear map.
Fusion Switch–MoE: A final Mixture of Experts layer adapts the fusion rule dynamically, selecting the best expert for the specific combination of motion and impact patterns.

3. Key Contributions

Novel Architecture: A two-stream framework (Motion–Mamba + Impact–Griffin) that explicitly encodes the causal link between radar-observed collapse and floor impact, utilizing State-Space Models (Mamba) and Gated Recurrent Units (Griffin) for efficiency.
Alignment-Aware Fusion: A fusion strategy using cross-conditioning and low-rank bilinear interaction to handle timing drift and suppress confounders (like object drops) without heavy computational cost.
Edge Optimization: The model is designed for real-time execution on a Raspberry Pi 4B, achieving low latency and energy consumption.
New Benchmark Dataset: The authors constructed a synchronized dataset of >3 hours of mmWave radar and triaxial vibration data in a wet bathroom environment. It includes 8 scenarios (falls, walking, standing, object drops) with frame-level annotations and subject-independent splits.

4. Experimental Results

The system was evaluated on a full-scale bathroom mock-up with running water.

Performance Metrics (Test Set):
- Accuracy: 96.1%
- Precision: 94.8%
- Recall (Fall Detection): 88.0%
- Macro F1: 91.1%
- AUC: 0.968
Comparison with Baselines:
- Compared to the strongest baseline (FMCW fusion), the proposed model improved accuracy by 2.0% and fall recall by 1.3%.
- It significantly reduced latency from 35.9 ms to 15.8 ms.
- It lowered energy consumption per 2.56s window from 14,200 mJ to 10,750 mJ.
Robustness: The model maintained high performance (>84% recall) even under synthetic time shifts (±150 ms) and modality dropouts (30% data loss).
Ablation Study: Confirmed that long-range temporal modeling (Mamba) primarily boosts recall, while cross-conditioning and bilinear fusion primarily boost precision by filtering false positives.

5. Significance

This work addresses a critical gap in elderly care by providing a privacy-preserving, non-intrusive, and energy-efficient solution for bathroom fall detection.

Practicality: By demonstrating successful deployment on a low-cost Raspberry Pi 4B, the research moves beyond theoretical models to viable edge computing solutions.
Reliability: The explicit modeling of the causal motion-impact relationship significantly reduces false alarms caused by common bathroom activities (dropping soap, walking), a major failure point in previous systems.
Benchmarking: The release of a synchronized, multi-scenario dataset with running water conditions provides a rigorous standard for future research in multimodal fall detection.