First International StepUP Competition for Biometric Footstep Recognition: Methods, Results and Remaining Challenges

The First International StepUP Competition leveraged the newly released UNB StepUP-P150 dataset to advance biometric footstep recognition, culminating in a global contest where the top team achieved a 10.77% equal error rate while highlighting persistent challenges in generalizing to unfamiliar footwear.

The Gist

Imagine you are walking through a busy airport. Usually, security checks your face or scans your fingerprint. But what if the security system could recognize you just by how you walk? Specifically, by the unique pressure pattern your feet make on the floor, like a hidden signature left in the dust.

This is the idea behind Footstep Biometrics. It's a new kind of security technology that doesn't require you to stop, look at a camera, or press a finger on a scanner. You just walk, and the floor knows who you are.

However, there's a big problem: People change how they walk.

• If you wear heavy boots instead of your running shoes, your steps change.
• If you are in a hurry (fast) or tired (slow), your steps change.
• If you are carrying a heavy bag, your steps change.

For a computer to recognize you, it needs to learn your "walking signature" from just a few samples (like a quick enrollment). But then, it has to recognize you later, even if you've changed your shoes or your speed. This is like trying to recognize a friend's handwriting even if they are writing with their non-dominant hand or using a different pen.

The Big Race: The StepUP Competition

To solve this, the researchers at the University of New Brunswick created a massive library of footstep data called StepUP-P150. It contains over 200,000 footprints from 150 different people, wearing 300 different pairs of shoes, walking at different speeds.

To see if anyone could crack the code, they launched the First International StepUP Competition. It was like a "Olympics for AI," where teams from universities and tech companies around the world tried to build the smartest "footstep detective."

The Challenge:
The teams were given the big library of data to study. Then, they were tested on a secret group of people.

• The Test: "Is this person who they say they are?"
• The Twist: The test included tricky scenarios. Some people were wearing shoes the AI had never seen before. Some were walking super fast or super slow. The AI had to guess correctly even when the conditions were totally different from what it learned.

The Winners and Their Secret Weapons

Out of 23 teams, three stood out. Here is how they did it, explained simply:

1. The Winner: Saeid UCC (Ireland) - The "Auto-Coach"

• The Problem: Building a perfect AI is hard. You have to choose the right "brain structure" (architecture) and the right "training settings" (hyperparameters). Usually, humans guess and check, which takes forever.
• The Solution: They built an AI Coach (called a Generative Reward Machine). Imagine a coach who watches a student athlete practice for just 5 minutes and can predict, "If you keep training this way, you will win the gold medal!"
• How it worked: The coach didn't wait for the AI to finish training. It looked at the early signs of learning and instantly decided, "This setup is great, keep going!" or "This one is failing, stop and try something else." This saved huge amounts of time and found the perfect combination of settings automatically.
• Result: They got the best score, making only about 11 mistakes out of 100 tricky tests.

2. The Runner-Up: Peneter ML (Iran) - The "Smart Simulator"

• The Problem: Testing every possible setting on the full, high-quality data is too expensive and slow (like trying to learn to drive by only driving on real highways).
• The Solution: They used a Transfer Learning trick. First, they trained their AI on "cheap" data (simplified versions of the footprints or fewer people). It was like practicing on a driving simulator first.
• How it worked: Once the AI learned the basics on the simulator, they "transferred" that knowledge to the real, high-quality data. This gave the AI a "warm start," so it didn't have to learn everything from scratch.
• Result: Very close to the winner, proving that smart shortcuts work.

3. Third Place: CyberTI (Australia) - The "Curriculum Teacher"

• The Problem: AI often gets overwhelmed if you throw all the hard data at it at once.
• The Solution: They created an Evolutionary Curriculum. Imagine a teacher who doesn't give a student a calculus textbook on day one. They start with simple addition, then move to algebra, then calculus.
• How it worked: Their AI started by learning only from easy walking conditions (barefoot, normal speed). As it got better, the "curriculum" automatically added harder conditions (heavy boots, running fast). The AI evolved its own training schedule to learn the hardest lessons only when it was ready.
• Result: A very strong performance, showing that how you teach the AI matters just as much as the AI itself.

The Big Takeaway: The "Shoe Problem"

The competition was a huge success, but it also revealed a stubborn problem.

• When the shoes were familiar: The AI was amazing. It could recognize people with near-perfect accuracy.
• When the shoes were new: The AI struggled. If a person wore a pair of shoes the AI had never seen before, the error rate jumped significantly.

The Analogy:
Think of it like recognizing a friend's voice.

• If they call you on their usual phone, you recognize them instantly.
• If they call you on a brand new, weirdly distorted phone, or if they are whispering, you might think it's a stranger.

The AI is great at recognizing the "voice" (the walking style) but gets confused when the "phone" (the shoes) changes the sound. The biggest challenge for the future is teaching the AI to ignore the shoes and focus only on the person.

Why This Matters

This competition wasn't just about winning a trophy. It proved that:

1. Footstep recognition is possible and could be used in airports, banks, and secure buildings without annoying people.
2. Big, diverse data is key. You can't build a smart system with small, boring data. You need data that covers every possible shoe and speed.
3. Automation is the future. The winners didn't just tweak the code; they built systems that automatically figured out the best way to learn.

The researchers are now inviting everyone to keep trying to beat their scores. The goal is to create a security system that knows you by your walk, no matter what shoes you're wearing or how fast you're running.

Technical Summary

1. Problem Statement

Biometric footstep recognition identifies individuals based on unique pressure patterns under their feet during walking. While promising for security and access control due to its passive nature (no user cooperation required), real-world deployment is hindered by high intra-subject variability.

• Core Challenge: Systems must generalize to unseen conditions (new users, different footwear, varying walking speeds) while relying on limited enrollment data (often just one shoe type and speed).
• Data Gap: Progress has been stalled by a lack of large, diverse public datasets. Existing datasets (e.g., CASIA-D, GaitRec) suffer from small participant numbers, limited variability, or lack of comprehensive labeling.
• Goal: To establish a robust benchmark for deep learning models to handle domain shifts caused by footwear and gait variations, moving from controlled lab environments to "in-the-wild" scenarios.

2. Methodology and Dataset

The competition was built around the UNB StepUP-P150 dataset, the largest and most comprehensive collection of high-resolution footstep pressure recordings to date.

The Dataset (StepUP-P150)

• Scale: ~200,000 footsteps from 150 individuals.
• Variability: 4 footwear conditions (Barefoot/Sock, Standard Sneakers, 2 pairs of Personal Shoes) × 4 walking speeds (Preferred, Slow-to-stop, Slow, Fast).
• Data Format: 3D pressure maps ($101 \times 75 \times 40$tensors representing time, height, width) captured via a piezoresistive sensor grid ($4 \text{ sensors/cm}^2$).
• Competition Split:
  • Training Set: Full dataset from 150 users (all conditions).
  • Reference Set (Enrollment): 15 users, 10 footsteps each (1 speed, 1 personal shoe).
  • Probe Set (Testing): 10,000 footsteps from 30 users (15 enrolled, 15 held-out impostors). Crucially, the probe set included unseen conditions (different shoes, different speeds) for the enrolled users to test generalization.

Competition Task

Teams were tasked with verification: determining if a probe footstep matches a claimed identity based on the limited reference data.

• Submission: Teams submitted similarity scores (0–1) for all 10,000 probes and a decision threshold.
• Baseline: A Python starter kit provided a Residual (2+1)D CNN (R(2+1)D) trained with supervised contrastive loss, achieving a baseline Equal Error Rate (EER) of 19.5%.

3. Key Contributions & Winning Approaches

The competition attracted 23 teams. The top three solutions all utilized advanced Automated Machine Learning (AutoML) and optimization strategies rather than relying solely on novel network architectures.

1st Place: Saeid UCC (Generative Reward Machine - GRM)

• Strategy: Developed an autonomous agent using a Generative Reward Machine (GRM) to jointly optimize neural network architecture and hyperparameters.
• Mechanism:
  • Phase 1 (Offline): Trained regression models to predict final performance based on early training dynamics (loss curves) for different architectures (ViT, Swin, ConvNeXt, R(2+1)D).
  • Phase 2 (Online): Used Reinforcement Learning (Q-learning) to search the space. The agent selected architectures and hyperparameters, trained for a few epochs, extracted dynamic features, and used the Phase 1 estimator to predict the reward, pruning unpromising trials.
• Result: Selected a lightweight R(2+1)D model with specific hyperparameters (TripletMarginLoss, Adam optimizer).
• Performance: EER 10.77%.

2nd Place: Peneter ML (Transfer-Initialized Multi-Fidelity Bayesian Optimization - TI-MFBO)

• Strategy: Addressed the "cold start" problem in Bayesian Optimization.
• Mechanism:
  • Transfer Learning: Pre-trained a surrogate model (Gaussian Process) on cheaper proxy tasks (e.g., 2D-CNN on max-intensity projections or downsampled data).
  • Multi-Fidelity: Dynamically allocated resources. Low-fidelity evaluations (fewer epochs/participants) filtered out poor configurations; only promising ones were promoted to high-fidelity evaluation.
• Result: Optimized the R(2+1)D network.
• Performance: EER 11.33%.

3rd Place: CyberTI (Evolutionary Curriculum Co-Optimization - ECCO)

• Strategy: Simultaneously optimized model hyperparameters and the training data curriculum.
• Mechanism: An evolutionary algorithm where "individuals" encoded both hyperparameters (learning rate, layers) and a curriculum schedule (sequence of data complexity). The model started with simple conditions (e.g., barefoot, preferred speed) and progressively introduced complex variables (different shoes, speeds).
• Result: Optimized R(2+1)D with TripletMarginLoss.
• Performance: EER 11.50%.

4. Results and Analysis

Overall Performance

• Top Teams: The top 5 teams achieved EERs between 10.77% and 12.23%, significantly outperforming the baseline (19.5%).
• Trade-offs: The winning model (Saeid UCC) favored low False Match Rates (FMR), making it more secure but slightly less convenient (higher False Non-Match Rate) compared to other teams.

Generalization Analysis (The "Footwear Problem")

A critical finding was the disparity in performance based on the type of variability:

• Walking Speed: Models generalized well to unseen speeds. EERs for slow/fast walking were comparable to or better than preferred speed (EER ~1.9%–4.6%).
• Footwear: Generalization to unfamiliar footwear was the primary failure point.
  • Personal Shoe #2 (Unseen): EERs spiked to 14.7% – 19.7%.
  • Standard Sneakers: EERs were lower (10.2% – 14.6%), likely because this shoe type was present in the training data for all 150 users, allowing the model to learn robust features for it.
• Failure Cases: Analysis of misclassified samples revealed that 650 footsteps were misclassified by all top three teams. These were predominantly linked to specific footwear styles (e.g., Birkenstock sandals) that created significant domain shifts from the reference data.

Architectural Trends

Despite exploring Transformers (ViT, Swin) and LSTMs, all top teams converged on the R(2+1)D CNN as the backbone. This suggests that for aligned, high-resolution pressure maps, a well-tuned, relatively simple 3D convolutional architecture is more effective than complex, parameter-heavy models, provided it is paired with robust optimization and data augmentation.

5. Significance and Future Directions

• Benchmark Establishment: The competition successfully established the first rigorous benchmark for pressure-based footstep biometrics, highlighting the gap between lab performance and real-world robustness.
• Data-Driven Optimization: The success of the top teams demonstrates that automated optimization (AutoML, curriculum learning) is currently more effective than architectural innovation for this specific modality. The diversity of the StepUP-P150 dataset allowed these methods to find robust solutions without overfitting.
• Remaining Challenge: The primary bottleneck remains footwear invariance. Current systems struggle to generalize to personal shoes not seen during enrollment.
• Future Work: The authors suggest future research should focus on:
  • Developing architectures and loss functions specifically designed to promote invariance to covariates like footwear.
  • Combining deep metric learning with advanced data augmentation techniques.
  • Continuing to expand datasets to include even more diverse footwear and environmental conditions.

In conclusion, the StepUP competition validated the potential of deep learning for footstep biometrics while clearly delineating the path forward: the field must move beyond standard recognition tasks to solve the specific challenge of cross-footwear generalization to achieve practical, deployable security systems.