Channel-Free Human Activity Recognition via Inductive-Bias-Aware Fusion Design for Heterogeneous IoT Sensor Environments

1. Problem Statement

Human Activity Recognition (HAR) in Internet of Things (IoT) environments faces a critical scalability challenge: sensor heterogeneity. Real-world deployments vary significantly in:

• Sensor Modalities: Accelerometers, gyroscopes, magnetometers, heart rate monitors, etc.
• Body Locations: Wrist, chest, ankle, waist, etc.
• Channel Configurations: The number of channels, their order, and their semantic meaning differ across datasets and devices.

The Limitation of Current Approaches:
Most existing HAR models operate under a "channel-fixed" assumption. They are designed for a specific input structure (e.g., a fixed number of channels in a fixed order). Consequently:

• A model trained on a 3-axis accelerometer dataset cannot be directly applied to a dataset with 9 channels (3-axis accelerometer + 3-axis gyroscope + 3-axis magnetometer) without redesigning the input layer.
• This prevents the creation of Foundation Models for HAR (large-scale, pre-trained models transferable across tasks), as the architecture cannot absorb diverse data distributions without retraining or structural modification.

The Goal:
To develop a Strict Channel-Free HAR framework where a single shared model can perform inference on heterogeneous sensor inputs (varying numbers, orders, and types of channels) without relying on predefined channel templates or sensor-specific input layers.

---

2. Methodology

The authors propose a framework based on Late Fusion (LF) combined with Inductive-Bias-Aware Design. The core architecture consists of three main components:

A. Channel-Wise Encoding with a Shared Encoder

Instead of concatenating all channels at the input (Early Fusion) or mixing them in intermediate layers (Middle Fusion), the proposed model processes each channel independently using a single shared encoder ($f_\theta$).

• Input Representation: The input is treated as a set of time series $\{X_i\}_{i=1}^c$, where $c$ is the variable number of channels.
• Mechanism: Each channel $X_i$ is fed into the same shared backbone (e.g., ResNet10) to produce a feature vector $z_i$.
• Benefit: This makes the model permutation-invariant and scalable; adding a new channel requires no architectural changes, only the processing of that specific channel through the shared encoder.

B. Metadata-Conditioned Late Fusion

Purely independent channel processing risks losing the structural relationships between sensors (e.g., that an accelerometer on the "left wrist" is semantically different from one on the "right ankle"). To recover this information without fixing channel order, the authors inject metadata via Conditional Batch Normalization (CBN).

• Metadata Embedding: For each channel, discrete identifiers are embedded:
  • Body Location ($p_i$)
  • Left/Right ($\ell_i$)
  • Sensor Type ($s_i$)
  • Axis ($a_i$)
• Conditional Batch Normalization (CBN): The metadata vector $m_i$ modulates the intermediate activations of the shared encoder.
  • Equation: $CBN(u; m_i) = (1 + \lambda_\gamma \tanh(\gamma(m_i))) \odot BN(u) + \beta(m_i)$.
  • Robustness: The use of $\tanh$ ensures that if metadata is missing or noisy (e.g., padding tokens), the modulation strength approaches zero, and the layer gracefully degrades to standard Batch Normalization.

C. Joint Supervision via Combination Loss

To ensure that individual channels remain discriminative while the final fused prediction is accurate, the model uses a Combination Loss:
$$\mathcal{L}_{comb} = \lambda \mathcal{L}_{fused} + (1 - \lambda) \mathcal{L}_{dist}$$

• $\mathcal{L}_{fused}$: Cross-entropy loss on the final prediction (obtained by mean-pooling all channel features).
• $\mathcal{L}_{dist}$: Cross-entropy loss on auxiliary predictions made from each individual channel's feature vector.
• Effect: This acts as an ensemble-style training mechanism, forcing the shared encoder to learn representations that are useful both individually and collectively.

---

3. Key Contributions

1. Formalization of Strict Channel-Free HAR: Defined a problem setting where a single model handles variable channel counts, orders, and semantics without input-dependent preprocessing (unlike previous "variable-format" works that still required linear transformations per input).
2. Inductive-Bias-Aware Architecture: Demonstrated that Late Fusion is the optimal inductive bias for channel-free settings. The proposed design explicitly recovers structural information lost by independent processing through metadata conditioning and joint supervision.
3. Comprehensive Evaluation: Validated the approach across:
   • 6 Datasets: PAMAP2, DSADS, MHEALTH, UCI-HAR, UniMiB SHAR, WISDM.
   • 5 Backbone Architectures: ResNet10, TCN, VGG16, MobileNetV2, EfficientNetB0.
   • Robustness Tests: Channel shuffling, missing channels, and signal-metadata inconsistency.
   • Transfer Learning: Cross-dataset pretraining and fine-tuning.

---

4. Experimental Results

The proposed method ("Ours") was compared against a Channel-Fixed Baseline, Early Fusion (EF), Middle Fusion (MF), and standard Late Fusion (LF).

• Robustness to Perturbations:
  • Channel Shuffling: The Baseline performance collapsed (e.g., 9% accuracy on PAMAP2) when channel order was randomized. The proposed method remained highly robust (89% accuracy), proving its order-invariance.
  • Missing Channels: The proposed method maintained high accuracy even when 50% of channels were dropped, significantly outperforming the Baseline.
• Clean Accuracy:
  • On the PAMAP2 dataset, the proposed method achieved 92.0% accuracy, outperforming the Channel-Fixed Baseline (80.3%) and standard LF (83.4%).
  • This indicates that the channel-free design does not sacrifice performance for flexibility; in fact, it improves it.
• Cross-Dataset Transfer:
  • Models pre-trained on heterogeneous datasets (e.g., mHealth or DSADS) and fine-tuned on PAMAP2 outperformed models trained from scratch.
  • The framework successfully transferred knowledge across datasets with different channel counts (e.g., 23ch to 40ch) without architectural redesign.
• Efficiency:
  • While the proposed model has slightly higher computational cost (MACs) due to per-channel processing, it remains efficient (inference in milliseconds) and scales well with GPU parallelism.

---

5. Significance and Conclusion

This paper establishes a practical pathway toward Foundation Models for Human Activity Recognition.

• Scalability: By removing the dependency on fixed channel structures, the proposed framework allows for the aggregation of diverse IoT datasets for large-scale pretraining, a prerequisite for building generalizable HAR foundation models.
• Real-World Applicability: The model's robustness to missing sensors and shuffled channel orders makes it highly suitable for real-world IoT deployments where sensor configurations are dynamic and often imperfect.
• Design Principle: The study highlights that fusion design is the central issue in channel-free learning. It argues that Late Fusion combined with explicit metadata conditioning and joint supervision provides the necessary inductive bias to balance flexibility with high recognition accuracy.

In summary, the authors demonstrate that a carefully designed channel-free architecture not only solves the reusability problem of heterogeneous sensors but also enhances model robustness and performance compared to traditional fixed-channel approaches.