When Multi-Sensor Fusion Fails to Generalize: Cattle Posture Classification Under Animal-Level and Temporal Distribution Shift

Technical Summary: When Multi-Sensor Fusion Fails to Generalize: Cattle Posture Classification Under Animal-Level and Temporal Distribution Shift

Problem Statement
Automated cattle posture classification systems (distinguishing lying from standing) frequently report near-perfect accuracy (F1 > 0.9) in literature. However, these estimates often rely on evaluation protocols that allow data leakage (e.g., random splits where the same animal appears in both training and test sets) and fail to account for temporal distribution shifts. It remains unclear whether multimodal sensor fusion—combining movement data with physiological and environmental signals—improves generalization or leads models to rely on context-specific "shortcut" signals that fail when deployment conditions change (e.g., different years, herds, or environmental contexts).

Methodology
The study utilized a dataset from a pasture-based beef cattle herd collected over two consecutive years (2024 and 2025) in Germany.

• Data Modalities:
  • Movement: Collar accelerometers (IMU tick counts at six thresholds) and GPS-derived walking distance.
  • Physiological: Rumen bolus sensors (internal temperature and rumination signals) for a subset of animals.
  • Environmental: Weather station data (temperature, humidity, wind, solar radiation, ground temperature).
• Models: XGBoost served as the primary model, with Logistic Regression, Random Forest, and LSTM networks used as baselines.
• Evaluation Protocols: The authors assessed robustness under four increasingly stringent protocols:
  1. Random Observation-Level Splits: Conventional random train-test splits (allowing subject overlap).
  2. Grouped Stratified Splits: Animal-level splits within the same year (2024).
  3. Leave-One-Animal-Out (LOAO): Training on all but one animal, testing on the held-out individual (within-year).
  4. Cross-Year Temporal Transfer: Training on the full 2024 dataset and testing on an independent cohort from 2025 (introducing both animal-level and temporal distribution shifts).
• Analysis: Performance was measured using Macro-F1 and Balanced Accuracy. Explainability was assessed via SHAP (SHapley Additive exPlanations) values to identify feature dependencies. Distribution shift was quantified using PCA and a domain classifier (ROC-AUC) to measure separability between years.

Key Results

• Overestimation of Performance: Under random observation-level splits, the multimodal model achieved a Macro-F1 of 0.99. Under grouped animal-level splits within the same year, performance remained high (0.94). However, under LOAO evaluation, performance dropped (0.89), and under cross-year transfer, it collapsed to 0.49.
• Multimodal Fusion Reduces Robustness: Contrary to the hypothesis that additional data improves generalization, the multimodal models performed worse than the collar-only model under temporal shift. The collar-only model achieved a Macro-F1 of 0.54 in the cross-year test, outperforming all multimodal configurations (0.49).
• Shortcut Learning: SHAP analysis revealed that multimodal models persisted in relying heavily on rumen-bolus activity and environmental variables even when predictive performance deteriorated. These features acted as context-dependent proxies for the 2024 recording conditions rather than robust posture indicators.
• Distribution Shift: PCA and domain classification confirmed substantial covariate shifts between the 2024 and 2025 datasets. Notably, even movement-only features (collar data) exhibited significant distributional shifts between years, contributing to the performance drop.
• Individual Variability: High standard deviations in LOAO results indicated that aggregate metrics masked severe failures for specific individual animals, with some approaching chance-level performance.

Key Contributions

1. Evaluation Protocol Critique: The study demonstrates that commonly used evaluation protocols (random splits) substantially overestimate real-world generalization capabilities, particularly when temporal shifts are present.
2. Multimodal Robustness Paradox: The authors show that while multimodal sensor fusion improves within-dataset benchmark performance, it can reduce robustness under temporal distribution shift by encouraging reliance on unstable, context-specific correlations.
3. Explainability for Diagnostics: By applying XAI to distribution-shift analysis, the study identifies specific feature dependencies (shortcut learning) that remain influential despite performance degradation, providing a mechanism to diagnose model failure modes invisible to standard metrics.

Significance and Claims
The paper argues that benchmark accuracy alone is insufficient to assess deployment readiness in livestock monitoring. The findings highlight that:

• Real-world robustness requires evaluation protocols that explicitly test generalization across unseen animals and time periods.
• Multimodal fusion is not inherently beneficial; without architectural constraints or robustness-focused training, additional modalities may introduce "Clever Hans" effects where models exploit spurious correlations.
• Explainability analysis is a critical tool for revealing hidden model dependencies that lead to failure under covariate shift.

The authors conclude that livestock-monitoring research must shift from benchmark-centric evaluation to robustness-centered evaluation to ensure systems are reliable under realistic deployment conditions.