⚡ electrical engineering

Performance, Robustness, and Failure Analysis of Machine-Learning-Based Wi-Fi RSSI Indoor Positioning in Multi-Floor Environments

This paper proposes a reliability-aware hybrid framework that integrates triplet-based metric learning with Bayesian weighted k-nearest neighbor inference to significantly enhance the robustness and failure resilience of multi-floor Wi-Fi RSSI indoor positioning systems, demonstrating through comprehensive evaluation that such an approach effectively reduces catastrophic localization errors and floor misclassifications compared to traditional accuracy-centric methods.

Original authors: Md. Sahidur Rahman, Mohammed Morshed Rana, Arifa Sultana

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Md. Sahidur Rahman, Mohammed Morshed Rana, Arifa Sultana

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific room in a massive, multi-story office building, but you have no map and no GPS (because GPS doesn't work well inside). Instead, you are using your phone to "sniff" the Wi-Fi signals coming from routers (Access Points) scattered around the building. This is called Wi-Fi RSSI fingerprinting.

The problem is that Wi-Fi signals are messy. They bounce off walls, get blocked by people, and sometimes routers just stop talking to your phone for a moment. In a multi-story building, this gets even trickier because your phone might think you are on the 3rd floor when you are actually on the 2nd, or it might guess your location is 50 meters away when you are right next to the elevator.

Most previous research tried to make the "average" guess as good as possible. But in real life, the average doesn't matter as much as the worst-case scenario. If a system is usually right but occasionally sends you to the wrong floor or the wrong city block, that's a catastrophic failure.

This paper introduces a new "hybrid" system designed to be reliable rather than just "accurate on average." Here is how it works, using simple analogies:

1. The Two-Part Detective Team

The authors built a system that acts like a two-person detective team to solve the location puzzle:

Detective A (The Floor Specialist): This part uses a technique called Triplet Metric Learning. Imagine you have a photo album where you group photos by floor. This detective learns to recognize the "vibe" of a floor. Even if the Wi-Fi signal is fuzzy, this detective can tell, "This signal pattern feels like the 2nd floor, not the 4th." It creates a special mental map where floors are clearly separated, like distinct islands.
Detective B (The Horizontal Specialist): Once the floor is identified, this detective figures out exactly where you are on that floor. It uses a method called Bayesian Weighted K-Nearest Neighbor. Think of this as asking a group of neighbors, "Where am I?" But instead of just taking a simple vote, it asks, "How sure are you?" If a neighbor's signal is weak or noisy, the system trusts them less. If the signal is strong, it trusts them more.

2. The "Safety Net" (Soft Floor Fusion)

Here is the clever part. Sometimes, even Detective A isn't 100% sure which floor you are on.

Old Way: The system would just pick the "most likely" floor and lock you in there. If it was wrong, you'd be lost.
This Paper's Way: If the system is unsure, it doesn't panic. It keeps the top 3 or 4 possible floors in mind. It calculates your position for each of those floors and then blends them together based on how likely each floor is. It's like saying, "I'm 60% sure you're on the 2nd floor, 30% on the 3rd, and 10% on the 1st." It averages your location across these possibilities. This prevents the system from making a huge, catastrophic mistake just because it guessed the wrong floor.

3. Testing for "Disaster Scenarios"

The authors didn't just test how well the system works on a perfect day. They simulated "disaster scenarios" to see how robust it is:

The "Noisy Room" Test: They added fake static noise to the Wi-Fi signals to mimic a crowded, chaotic environment.
The "Broken Router" Test: They pretended that half the Wi-Fi routers in the building suddenly stopped working (a common real-world issue).

The Results:

Old Systems: When routers went missing or signals got noisy, old systems (like Random Forests or standard KNN) would often go crazy, guessing locations that were 50 to 200 meters away. They had "heavy tails," meaning they were usually okay but occasionally disastrous.
The New Hybrid System: Even when half the routers were "broken," this new system kept its worst-case errors very low (around 33 meters). It didn't just get the average right; it suppressed the disasters.

4. Why This Matters

The paper argues that for safety-critical things (like emergency response or guiding people in a building), you don't want a system that is "usually" right. You want a system that rarely makes a huge mistake.

Catastrophic Failure Rate: The new system reduced the chance of a massive error (like guessing the wrong floor by a wide margin) by about 95% compared to some older methods.
Confidence: The system knows when it is unsure. If it's not confident, it uses its "safety net" (the soft fusion) to stay safe, rather than guessing blindly.

Summary

Think of this paper as upgrading a GPS for indoor use. Old GPSs were like a friend who gives you directions that are usually correct but sometimes sends you to the wrong city. This new system is like a cautious, experienced guide who says, "I'm pretty sure we are on the 2nd floor, but if I'm wrong, I have a backup plan that keeps us from walking off a cliff." It prioritizes safety and reliability over just being "fast" or "average."

Technical Summary: Performance, Robustness, and Failure Analysis of Machine-Learning-Based Wi-Fi RSSI Indoor Positioning in Multi-Floor Environments

1. Problem Definition

Indoor positioning systems (IPS) based on Wi-Fi Received Signal Strength Indicator (RSSI) fingerprinting are widely used due to their low cost and ability to reuse existing infrastructure. However, in practical multi-floor environments, these systems face significant challenges:

Signal Instability: RSSI values fluctuate due to environmental dynamics, multipath propagation, and device diversity.
Infrastructure Degradation: Real-world deployments suffer from Access Point (AP) dropout, sparse signal visibility, and signal attenuation between floors.
Floor Ambiguity: Vertical signal attenuation often leads to floor misclassification. In multi-floor scenarios, an incorrect floor estimate propagates into catastrophic 3D localization errors, even if the 2D horizontal error appears acceptable.
Evaluation Gaps: Existing literature predominantly optimizes for median or mean localization accuracy. These metrics fail to capture "tail behavior" (extreme errors), catastrophic failure rates, or system robustness under degraded signal conditions. Consequently, systems with high nominal accuracy may still exhibit heavy-tailed error distributions and unreliable confidence estimates, rendering them unsuitable for safety-critical applications.

2. Methodology

The authors propose a reliability-aware hybrid indoor Wi-Fi RSSI positioning framework that decouples floor classification from horizontal localization while integrating uncertainty modeling. The system operates in two distinct phases: offline training and online inference, with no retraining during deployment.

A. Core Components:

Triplet-Based Metric Learning for Floor Classification:
- A neural network learns a discriminative embedding space using triplet loss.
- The objective is to minimize the distance between fingerprints from the same floor and maximize the distance between fingerprints from different floors.
- This creates a robust representation for floor inference, addressing the ambiguity caused by vertical signal attenuation.
- Floor prediction is performed using Bayesian K-Nearest Neighbor (BWKNN) in the embedding space to generate posterior probabilities and confidence scores.
Bayesian Weighted K-Nearest Neighbor (BWKNN) for Horizontal Localization:
- Horizontal positioning ( $x, y$ ) is performed in the original RSSI space using cosine similarity.
- Unlike deterministic KNN, this approach uses probabilistic weighting based on likelihood formulations to handle noisy or sparse measurements.
- The number of neighbors ( $K_{xy}$ ) and weight smoothness are adaptively adjusted based on the number of visible APs to mitigate the effects of AP dropout.
Confidence-Driven Soft Floor Fusion:
- A gating mechanism evaluates the confidence of the floor prediction.
- High Confidence: The baseline horizontal estimate is accepted.
- Low Confidence: The system performs "soft floor fusion." It calculates horizontal estimates for the top- $K$ candidate floors and fuses them using posterior-weighted averaging. This prevents catastrophic failures caused by a single wrong-floor classification.

B. Pre-processing:

Missing RSSI values are imputed with the receiver noise floor (-110 dBm).
Per-sample Z-score normalization is applied to reduce device bias.
A visibility mask is constructed to distinguish between weak signals and missing APs.

3. Key Contributions

The paper advances the field from an accuracy-centric paradigm to a reliability-oriented framework through four main contributions:

Hybrid Framework: Integration of deep metric learning for discriminative floor embeddings with Bayesian WKNN for uncertainty-aware horizontal positioning.
Reliability-Aware Inference: A strategy utilizing confidence-sensitive floor estimation and soft floor fusion to mitigate the propagation of wrong-floor errors.
Controlled Robustness Evaluation: Experiments simulating realistic stress conditions, including additive Gaussian RSSI noise and varying levels of AP dropout.
Failure-Aware Performance Analysis: An evaluation protocol extending beyond median error to include tail statistics (P90, P95), Catastrophic Failure Rate (CFR), wrong-floor penalties, and Risk-Coverage (RC) curves.

4. Experimental Results

The framework was evaluated on a public multi-floor dataset (Tampere University) comprising three floors and 446 reference points. It was compared against classical KNN, Hierarchical KNN, tree-based models (Random Forest, XGBoost, CatBoost), MLP, and standalone metric learning or Bayesian approaches.

Nominal Accuracy: The proposed hybrid model achieved a 3D median error of 6.64 m and a floor classification accuracy of 93.38% (Macro-F1: 0.927), which is competitive with state-of-the-art methods.
Tail Behavior and Catastrophic Failures:
- The hybrid model significantly suppressed extreme errors. Its 3D P90 error was 12.53 m, compared to 50.89 m for Random Forest and 14.88 m for Metric-WKNN alone.
- Catastrophic Failure Rate (CFR): At a 15 m error threshold, the hybrid model achieved a CFR of 2.44%, representing a 94.8% reduction compared to Random Forest (47.05%) and a 22.8% reduction compared to Bayesian WKNN. At a 20 m threshold, the CFR dropped to 0.10%.
Robustness to Signal Degradation:
- AP Dropout: Under extreme sparsity (50 visible APs), the hybrid model maintained a worst-case P90 of 32.85 m. In contrast, Bayesian WKNN failed catastrophically with a P90 of 266.79 m, and tree-based models exceeded 95 m.
- RSSI Noise: The hybrid model remained stable (P90 ~12–13 m) under noise perturbations up to 8 dB, whereas tree-based models and MLPs showed significant error inflation.
Risk-Coverage Analysis: The model achieved a Risk-Coverage Area Under Curve (RC-AUC) of 6.06, indicating superior uncertainty calibration and a smooth trade-off between prediction coverage and tail error compared to deterministic baselines.
Computational Efficiency: The inference time is approximately 1.16 ms per sample with a memory footprint of 0.94 MB, confirming suitability for real-time deployment.

5. Significance and Claims

The paper claims that average localization accuracy is insufficient for evaluating indoor positioning systems intended for real-world, safety-critical deployment. The primary significance of this work lies in demonstrating that:

Robustness is distinct from Accuracy: A system can have competitive median accuracy but fail catastrophically under signal degradation. The proposed hybrid approach explicitly targets the suppression of these tail errors.
Uncertainty Modeling is Critical: By incorporating probabilistic inference and confidence-aware gating, the system can detect ambiguous situations (e.g., floor confusion) and degrade gracefully rather than producing large, unbounded errors.
Failure-Aware Benchmarking is Necessary: The authors argue that future research must adopt metrics such as CFR, P90, and wrong-floor penalties to ensure the dependability of IPS in complex, multi-floor environments.

The study concludes that while deep learning and metric learning improve feature representation, combining them with probabilistic inference (Bayesian WKNN) and specific failure-mitigation strategies (soft fusion) is essential for creating reliable, real-world indoor positioning systems that do not require additional hardware.