Activity Recognition from Smart Insole Sensor Data Using a Circular Dilated CNN

This paper presents a circular dilated convolutional neural network (CDCNN) for real-time activity recognition using multi-modal smart insole sensor data, which achieves 86.42% accuracy in a subject-independent four-class classification task, demonstrating that inertial sensors are the primary contributors to performance while remaining suitable for embedded deployment.

The Gist

Imagine your shoes are secretly talking to a computer, whispering every step you take, every time you sit down, and even how you balance on one foot. That's exactly what this paper is about: teaching a computer to understand human movement by listening to "smart insoles" hidden inside your shoes.

Here is the story of how the researchers taught a computer to do this, explained without the heavy jargon.

The Problem: Shoes with Superpowers

Think of a regular shoe as a silent partner. A smart insole, however, is like a shoe with a tiny, high-tech detective team inside. It has two types of spies:

1. Pressure Sensors: These are like sensitive fingertips spread across the bottom of the shoe. They feel exactly where your weight is pressing down (heel, toe, arch).
2. Motion Sensors (Accelerometers & Gyroscopes): These are like the inner ear of a gymnast. They feel how fast you are moving, which way you are tilting, and how you are spinning.

The goal? To figure out what you are doing just by listening to these sensors. Are you standing still? Walking? Sitting? Or doing a tricky "tandem" stance (balancing heel-to-toe)?

The Solution: The "Circular Time-Traveling" Brain

The researchers built a special computer brain called a Circular Dilated CNN. That sounds scary, but let's break it down with an analogy:

• The CNN (Convolutional Neural Network): Imagine you are looking at a long strip of film showing your foot moving. A standard camera might look at one frame at a time. This brain looks at the whole strip, noticing patterns like "heel hits, then toe pushes."
• Dilated (The "Zoom Out" Lens): Usually, to see a long pattern, you need a huge camera lens. But this brain uses a "dilated" lens. It's like looking at a long road through a telescope that skips a few inches between each glance. This allows it to see the whole story of a step (from start to finish) without needing a massive, heavy brain.
• Circular (The "Infinite Loop"): This is the cleverest part. When you walk, your steps are a loop. If you cut a video of your walk in half, the end of one clip might look weird because the foot is in mid-air. The "circular" trick tells the computer: "If the foot is at the end of the clip, pretend it's wrapping around to the beginning." This stops the computer from getting confused by the edges of the data.

The Training: Learning from a Crowd

The researchers didn't just teach the computer with one person's data. They used data from 14,748 different "moments" collected from many different people.

To make sure the computer is truly smart and not just memorizing answers, they played a strict game:

• The Test: They taught the computer using data from 11 people.
• The Exam: They then tested it on 3 completely new people it had never seen before.

This is like teaching a student with a textbook, then giving them a test with questions from a different author to see if they actually understand the concept of walking, rather than just memorizing the specific steps of their teacher.

The Results: The Computer vs. The Human Expert

The computer brain (the CDCNN) got 86.4% of the answers right on the new people. That's pretty good!

However, they also tried a different, older method called XGBoost (think of this as a very organized, super-fast human expert who looks at a spreadsheet of all the numbers). The human expert got 87.8% right.

Why did the older method win slightly?
The older method looked at every single number at once, like a detective reading a whole novel in one glance. The new computer brain looked at the story in chunks. The new brain is slightly less accurate on this specific test, BUT it has huge advantages:

1. It's faster: It can run on a tiny chip inside the shoe in real-time.
2. It's flexible: It understands the flow of movement, not just a snapshot.
3. It's explainable: We can see exactly when it made a decision.

The Big Discovery: What Matters Most?

The researchers asked the computer: "Which sensor is doing the heavy lifting?"

They used a trick called Permutation Importance. Imagine you have a team of 24 spies. To see who is most important, you blindfold one spy (shuffle their data) and see if the team fails.

• The Result: The motion sensors (accelerometer and gyroscope) were the MVPs. They were the most critical for telling the difference between standing and walking.
• The Pressure Sensors: These were the support team. They were crucial for knowing where the foot was planted, especially for balancing acts.

Why Should You Care?

This isn't just about counting steps. This technology could:

• Help the elderly: Detect if a grandparent is about to fall before they even hit the ground.
• Fix your posture: Tell a physical therapist if a patient is walking correctly during rehab.
• Be invisible: Unlike cameras (which feel creepy) or radar (which needs big antennas), this is just a shoe. It's private, comfortable, and works in the dark.

In a nutshell: The researchers built a smart shoe brain that understands how we move by looking at the "shape" of our steps over time. It's a bit less accurate than a spreadsheet expert right now, but it's faster, smaller, and ready to live inside your shoe to help you stay safe and healthy.

Technical Summary

1. Problem Statement

The paper addresses the challenge of human activity recognition (HAR) using smart insoles equipped with multi-modal sensors (pressure mats, accelerometers, and gyroscopes).

• Context: While existing methods often rely on hand-crafted features or treat sensor modalities (pressure vs. inertial) independently, there is a need for an end-to-end deep learning approach that fuses all available channels from a smart insole.
• Constraints: The solution must be suitable for embedded deployment (low latency, real-time inference) and robust to subject variability (generalizing to new users without retraining).
• Goal: To classify four specific activities—Standing, Walking, Sitting, and Tandem stance—using raw time-series data from a 160-frame window containing 24 sensor channels.

2. Methodology

A. Dataset and Preprocessing

• Source: The study utilizes the STEP-Insole Dataset, comprising recordings from multiple subjects performing the four target activities.
• Data Structure:
  • Input: Fixed-length windows of 160 time steps.
  • Channels (24 total): 18 pressure sensors, 3-axis accelerometer, and 3-axis gyroscope.
  • Dimensions: Input matrices are shaped as $(N, T, F) = (N, 160, 24)$.
  • Split: A strict subject-independent split was enforced (no subject overlap between training, validation, and test sets) to ensure the model generalizes to unseen users.
    • Train: 11 subjects (14,047 samples)
    • Validation: 2 subjects (3,333 samples)
    • Test: 3 subjects (3,689 samples)

B. Proposed Model: Circular Dilated CNN (CDCNN)

The authors propose a Circular Dilated 1D Convolutional Neural Network designed to process the temporal sequence of sensor data directly.

• Architecture Components:
  1. Circular Dilated Convolution Blocks: Four stacked blocks using dilation rates of $d = 2^i$ (1, 2, 4, 8).
     • Dilated Convolutions: Expand the effective temporal receptive field exponentially without increasing parameters, allowing the model to capture both short-term foot contacts and longer-term gait patterns.
     • Circular Padding: Applied to avoid boundary artifacts at the start/end of segments, crucial when activity transitions do not align perfectly with window borders.
     • Layers: Each block includes 1D Conv (kernel size 3, 64 filters), Batch Normalization, and ReLU activation. Dropout (0.2) is applied after each block.
  2. Classification Head:
     • Global Average Pooling: Reduces the temporal dimension to a single vector.
     • Fully Connected Layer: Maps the 64-dimensional representation to 4 output logits (classes).
• Training: Optimized using Adam (lr=0.01) with Cross-Entropy loss and Early Stopping (patience=20).

C. Feature Importance Analysis

To interpret the model and understand sensor contributions, the authors employed Permutation Feature Importance:

• Method: Randomly shuffling values of a specific channel across the test set and measuring the drop in accuracy ($I_f = A_{base} - A_{perm}$).
• Comparison: This analysis was performed on both the CDCNN and a baseline XGBoost model trained on flattened data.

3. Key Contributions

1. Novel Architecture: Introduction of a CDCNN specifically tailored for smart insole data, utilizing circular padding and dilated convolutions to handle temporal dependencies and boundary conditions effectively.
2. End-to-End Fusion: A unified model that processes raw multi-modal data (pressure + IMU) without manual feature engineering.
3. Subject-Independent Evaluation: Rigorous testing on a held-out set of subjects, demonstrating the model's ability to generalize to new users.
4. Interpretability: Comprehensive analysis of sensor contributions via permutation importance, revealing the relative roles of inertial vs. pressure sensors.
5. Embedded Suitability: The architecture is designed for low computational footprint, making it viable for real-time inference on embedded hardware within the insole.

4. Results

• Classification Performance:
  • CDCNN: Achieved 86.42% test accuracy on the subject-independent split.
  • XGBoost Baseline: Achieved 87.83% test accuracy on the same split.
  • Analysis: While XGBoost slightly outperformed the CDCNN (likely due to its ability to exploit non-linear interactions in flattened vectors and robustness to hyperparameter tuning), the CDCNN remains highly competitive.
• Feature Importance Findings:
  • Inertial Sensors: Accelerometer and gyroscope channels consistently showed the highest importance, indicating that motion dynamics are critical for distinguishing the four activities.
  • Pressure Sensors: While individual pressure sensors had lower importance than inertial ones, spatial patterns (specifically heel and toe regions) were significant. The combined pressure modality remains essential for characterizing stance and contact patterns.
  • Model Comparison: The CDCNN and XGBoost showed broadly similar feature rankings, though XGBoost assigned slightly higher relative importance to certain pressure channels, reflecting different inductive biases.

5. Significance and Conclusion

• Practicality: The CDCNN offers a real-time, low-latency alternative to tree-based ensembles, which is critical for embedded wearable devices where power and compute are limited.
• Interpretability: Unlike "black box" ensembles, the CNN's temporal feature maps can be inspected (e.g., via attention or class activation mapping) to provide time-resolved insights into when and how the model makes decisions.
• Future Directions: The authors suggest future work should focus on:
  • Leave-one-subject-out evaluation for stricter generalization testing.
  • Expanding to finer-grained activities (e.g., stair climbing, turning) and continuous gait event detection.
  • Model compression (pruning) for resource-constrained insoles.
  • Incorporating temporal smoothing or recurrent layers to handle label noise.

In summary, the paper demonstrates that a specialized circular dilated CNN is a viable, interpretable, and efficient architecture for multi-modal smart insole activity recognition, achieving performance comparable to strong gradient-boosted baselines while offering distinct advantages for real-time embedded deployment.