From Vision to Assistance: Gaze and Vision-Enabled Adaptive Control for a Back-Support Exoskeleton

Technical Summary: From Vision to Assistance: Gaze and Vision-Enabled Adaptive Control for a Back-Support Exoskeleton

Problem Statement
Back pain remains a prevalent work-related disorder, with active back-support exoskeletons proposed as a solution to redistribute spinal loads. However, the effectiveness of active exoskeletons critically depends on providing timely, context-aware assistance. Existing control strategies typically rely on reactive user signals (e.g., IMU, EMG, or applied torque) or static vision systems that do not directly inform the control loop. These approaches often lack the ability to perceive the operative environment, leading to mistimed assistance that can reduce user trust, comfort, and long-term acceptance. Specifically, current systems struggle to distinguish between a user merely bending to look at an object versus actively grasping it, resulting in delays in torque delivery during critical lifting phases.

Methodology
The authors propose a vision-gated control framework for an active lumbar occupational exoskeleton that integrates egocentric vision with wearable gaze tracking to enable context-aware assistance. The system architecture comprises three main components:

1. Perception Module: Utilizing a first-person camera mounted on Tobii Pro Glasses 3, the system streams RGB frames and gaze coordinates. A fine-tuned YOLOv9 detector classifies objects as "Grasped" or "Not Grasped." By mapping the user's gaze fixation onto the image plane, the system generates binary indicators ($\chi^+_t$ and $\chi^-_t$) to determine if the user is fixating on a grasped object. A temporal filtering logic (sliding window with dwell ratio) converts these indicators into a robust binary vision gate ($\mathbb{1}_{box}$), ensuring transitions reflect intentional grasp events rather than transient fluctuations.

2. Finite-State Machine (FSM): The FSM governs the task progression through four states: (0) standing without a box, (1) bending to pick, (2) standing with a box, and (3) bending to place. Transitions are triggered by trunk inclination thresholds ($\theta_{stand}$, $\theta_{bend}$) and the vision gate. Crucially, the vision gate allows the system to autonomously trigger the transition from "bending to pick" to "standing with a box" (State 1 $\to$ 2) and manage the release (State 3 $\to$ 0) based on the grasp state, rather than relying solely on posture.

3. Variable Admittance Control: The controller delivers assistive torque ($\tau_{ass}$) via an admittance law. The reference torque is a weighted sum of trunk torque ($\tau_w$) and box torque ($\tau_{box}$). The inclusion of $\tau_{box}$ is gated by the vision module. The system employs a variable stiffness strategy, switching between "Soft" (backdrivable, $K=0$) and "Hard" (stiff, $K \ge K_{min}$) modes based on motion velocity, with smoothed transitions to ensure fluency.

Experimental Protocol
A user study was conducted with 15 healthy volunteers performing repeated stooping load-lifting trials with a 4 kg box. The study compared three conditions:

• EC1: No exoskeleton (baseline).
• EC2: Exoskeleton with posture-based control (no vision).
• EC3: Exoskeleton with vision-gated control.

Participants were unaware of the vision activation in EC2 vs. EC3 to avoid bias. Performance was evaluated via objective torque analysis and a 13-item Likert-scale questionnaire assessing physical demand, fluency, trust, and comfort.

Key Results

• Perception Accuracy: The YOLO-based grasp detection achieved a macro-averaged F1-score of 92.35% (Precision: 92.84%, Recall: 91.86%).
• Physical Demand: Statistical analysis (Friedman test, $p < 0.001$) revealed that the vision-gated condition (EC3) significantly reduced perceived physical demand compared to both the no-exoskeleton baseline and the posture-only exoskeleton (EC2). EC3 scores were notably higher (mean 3.60) than EC2 (mean 2.13).
• User Experience: For all subjective metrics (fluency, trust, comfort, mutual understanding), the vision-gated mode significantly outperformed the posture-only mode (Wilcoxon signed-rank tests, $p < 0.05$ after Holm correction). Effect sizes were large (Kendall's $W = 0.752$ for physical demand; rank-biserial correlations often $\ge 0.80$).
• Control Performance: Quantitative analysis showed that vision gating enabled earlier and stronger assistance. The system reduced the lag between the actual grasp and torque onset, particularly during the critical transition from bending to lifting. Participants did not need to reach a specific posture threshold to receive support, as the system anticipated the task based on the grasp event.

Significance and Claims
The paper claims that integrating egocentric vision and gaze tracking into the control loop of an active back-support exoskeleton significantly enhances the system's responsiveness, ergonomics, safety, and user acceptance. By moving from purely reactive, posture-based control to proactive, context-aware assistance, the proposed framework addresses the "blindness" of current exoskeletons to the operative environment. The results suggest that vision-gated assistance improves the naturalness of human-robot collaboration, fostering greater trust and comfort, which are identified as critical factors for the industrial adoption of active exoskeletons.

Limitations and Future Work
The authors acknowledge that the study was conducted under controlled laboratory conditions with a single box type and known weight. They note that future work must address variable payloads, potentially through online weight estimation or dynamic adaptation. Additionally, the current study focused on lumbar assistance; future iterations aim to integrate coordinated control of lumbar and hip joints to support diverse tasks like squatting and walking.