Understanding Behavioural Risk Signals in Socio-Technical Work Systems Using Explainable Machine Learning

Technical Summary: Understanding Behavioural Risk Signals in Socio-Technical Work Systems Using Explainable Machine Learning

Problem Statement
Conventional research into occupational safety often models accident risk as a linear function of aggregated safety climate perception scores. The authors argue that this approach potentially obscures behavioural heterogeneity and critical interaction effects embedded at the item level. Within socio-technical work systems, accidents rarely stem from single technical failures but emerge from complex interactions among organisational structures, supervisory practices, and workforce characteristics. The study posits that occupational risk may arise from nonlinear behavioural configurations shaped by organisational signals, which average-based models fail to detect. Furthermore, a persistent empirical observation suggests that accident occurrence is unevenly distributed even among workers with similar hazard exposures, indicating that aggregated models may mask meaningful variations in how safety signals are interpreted and translated into behaviour.

Methodology
The study employed an observational design using data from 207 workers in a safety-critical industrial environment. The analytical unit was the individual worker, with the outcome variable defined as a binary indicator of whether a work-related accident occurred in the previous 12 months.

• Data Collection: A structured self-administered questionnaire captured behavioural safety indicators, safety perceptions, and demographics. Crucially, the study retained predictors at the individual item level rather than aggregating them into composite dimension scores, to preserve perceptual granularity and avoid aggregation bias.
• Modelling Strategy: The research implemented a comparative analytical framework:
  1. Logistic Regression: Used as a baseline linear model to estimate directional associations and odds ratios.
  2. Ensemble Machine Learning: Random Forest and Gradient Boosting classifiers were employed to capture nonlinear patterns and higher-order interactions. Hyperparameters were optimized via cross-validated grid search.
  3. Explainable AI (XAI): SHAP (SHapley Additive exPlanations) values, specifically the TreeSHAP algorithm, were applied to the ensemble models to decompose predictions and quantify the marginal contribution of each feature. This allowed for both global feature ranking and local interpretation of interaction structures.
• Validation: Models were evaluated using stratified 10-fold cross-validation on independent test sets. Performance was assessed via ROC-AUC, accuracy, precision, recall, F1-score, and Matthews Correlation Coefficient (MCC). Robustness was verified through repeated cross-validation and sensitivity analyses regarding classification thresholds.

Key Results

• Model Performance: Ensemble learning models significantly outperformed the linear baseline. Gradient Boosting achieved the highest discriminative performance (AUC = 0.837, Accuracy = 0.773), followed by Random Forest (AUC = 0.841, Accuracy = 0.744). Logistic regression demonstrated moderate discrimination (AUC = 0.757, Accuracy = 0.725).
• Linear vs. Nonlinear Findings: Logistic regression identified significant directional associations for specific variables, including gender, work experience (10–20 years), safety training participation, and employment type. However, aggregated safety climate dimensions and total scores showed no significant predictive contribution in the linear model.
• Nonlinear Structures and Thresholds: The ensemble models, interpreted via SHAP, revealed that accident risk is concentrated within specific perceptual configurations rather than distributed linearly.
  • Management Responsiveness: Items related to management responsiveness (e.g., r2_1, r2_4) were the most influential predictors. SHAP dependence plots indicated threshold-like behavior, where risk probability shifted abruptly once specific perception levels were exceeded, rather than changing gradually.
  • Training and Experience: Safety training participation acted as a "behavioural stabilizer," narrowing the dispersion of risk configurations rather than uniformly reducing probability. Interactions between management responsiveness and worker experience showed that experienced workers formed tighter behavioural clusters, suggesting experience-mediated adaptation.
  • Asymmetry: SHAP analysis revealed asymmetric contributions, where similar increases in perception scores did not translate into equivalent risk reductions across all workers, supporting the existence of behavioural tipping points.

Key Contributions

1. Item-Level Granularity: The study demonstrates the value of retaining safety climate indicators at the item level. Aggregating these into composite scores appears to dilute critical item-level signals regarding managerial interaction and trust calibration, obscuring the nonlinear mechanisms driving accident risk.
2. Configuration-Sensitive Analytics: By integrating ensemble learning with XAI, the research identifies that accident risk is a configuration-dependent phenomenon. It highlights specific combinations of management responsiveness, training engagement, and worker experience that create high-risk behavioural boundaries, which linear models fail to capture.
3. Interpretability in Safety Management: The application of SHAP values translates complex predictive outputs into interpretable behavioural insights. This allows for the identification of asymmetric predictor contributions and interaction boundaries, moving beyond opaque "black box" predictions to actionable safety intelligence.

Significance and Claims
The paper claims that its findings challenge the sufficiency of population-average safety climate interpretations in occupational safety. It argues that accident risk is not merely a function of the "strength" of a safety climate but is embedded within nonlinear behavioural thresholds and interaction structures.

The authors assert that their approach supports a shift from retrospective, average-based monitoring toward proactive, threshold-sensitive behavioural diagnostics. By identifying specific perceptual boundaries where accident probability changes disproportionately, organisations can potentially detect emerging instability in safety systems before incidents occur. The study concludes that while logistic regression remains useful for identifying directional associations, ensemble learning combined with explainable AI is superior for uncovering the complex, configuration-dependent mechanisms that govern safety behaviour in socio-technical work environments.

The authors maintain a modest stance, acknowledging that the identified thresholds are contingent on the observed data distribution and that external validation across different industries and cultural contexts is necessary. They do not claim universal superiority of machine learning over statistical methods but rather highlight that the performance advantage in this context arises from the presence of nonlinear behavioural complexity in the data.