Deep Reinforcement Learning-Enhanced Event-Triggered Data-Driven Predictive Control for a 3D Cable-Driven Soft Robotic Arm

Technical Summary: Deep Reinforcement Learning-Enhanced Event-Triggered Data-Driven Predictive Control for a 3D Cable-Driven Soft Robotic Arm

Problem Statement
Soft robotic arms possess inherent compliance and safety advantages but present significant control challenges due to their nonlinear, time-varying, and hysteretic dynamics. Traditional model-based approaches, such as Finite Element Methods (FEM) or Cosserat rod formulations, are often computationally prohibitive for real-time deployment on resource-limited platforms. Data-enabled predictive control (DeePC) offers a model-free alternative by leveraging measured input-output trajectories to construct predictive controllers without explicit system identification. However, standard DeePC operates in a receding-horizon manner, requiring the solution of a constrained optimization problem (typically a quadratic program) at every sampling instant. For soft robots demanding high sampling rates to capture fast transient behaviors, this repeated re-optimization imposes a substantial computational burden, limiting real-time applicability. While event-triggered control (ETC) has been explored to mitigate this in other frameworks, existing data-driven ETC methods typically rely on static, heuristic triggering rules (e.g., fixed error thresholds). These static rules struggle to balance computational savings with control performance in systems characterized by strongly nonlinear and time-varying dynamics, often resulting in either limited savings or degraded tracking accuracy.

Methodology
The authors propose an adaptive Reinforcement Learning-based Event-Triggered DeePC (RL-ET-DeePC) framework designed to reduce computational load while maintaining closed-loop performance. The methodology integrates three core components:

1. Dimensionality-Reduced DeePC: To address the high dimensionality of Hankel matrices required for soft robots, the framework employs Singular Value Decomposition (SVD). Offline data is collected to construct input-output Hankel matrices, which are then compressed by retaining only the dominant singular values. This reduces the decision variable dimension, making the online optimization more tractable. The controller solves a regularized optimization problem to minimize tracking error and control effort while handling measurement noise via slack variables.
2. RL-Based Event Triggering: Instead of re-optimizing at every time step, a Deep Reinforcement Learning (DRL) agent determines whether to invoke the DeePC optimizer. The agent operates within a Markov Decision Process (MDP) where:
   • State: Defined by the tracking error and its temporal variation.
   • Action: A binary decision ($0$ or $1$) to either skip optimization (reusing a buffered control input) or trigger a new optimization.
   • Reward: A function balancing tracking error penalties against the computational cost of triggering. A tunable parameter $\rho$ controls the trade-off between accuracy and efficiency.
   • Algorithms: The framework is algorithm-agnostic; the authors evaluate Value-based (DQN), Policy-gradient (PPO), and Actor-Critic (A2C) approaches.
3. Supervisory Override: To ensure robustness and prevent excessive open-loop execution, a supervisory layer forces re-optimization if the control buffer is empty or if the time elapsed since the last update exceeds the prediction horizon minus one. This guarantees a valid control sequence is always available.

Key Contributions

• Adaptive Event Triggering: The paper introduces a learned, state-dependent triggering mechanism that replaces static heuristic rules. This allows the controller to adaptively balance computational cost and tracking performance based on the current system dynamics.
• Integration of SVD and RL: The framework combines SVD-based dimensionality reduction with DRL to create a computationally efficient, model-free predictive controller suitable for resource-constrained soft robotic platforms.
• Zero-Shot Transfer: The study demonstrates the ability of the learned policy to transfer from simulation to hardware without retraining, achieving consistent performance on a physical 3D cable-driven soft robotic arm.

Results
The framework was validated through extensive simulations and hardware experiments on a custom-built 3D cable-driven soft robotic arm.

• Simulation Results: Compared to periodic DeePC, the RL-ET-DeePC framework (specifically using the PPO algorithm) reduced the optimization frequency by up to 66% (trigger rate dropping from 100% to ~34%) while maintaining comparable tracking accuracy (RMSE remained within acceptable ranges). The results showed that the learned policy concentrates re-optimization efforts during transient phases (rapid state changes) and suppresses updates during steady-state operation.
• Hardware Experiments: On the physical robot, the framework achieved a 34% reduction in optimization frequency compared to periodic DeePC. While the reduction was lower than in simulation (attributed to modeling mismatches, friction, and hysteresis in the real system), the tracking accuracy remained comparable to the periodic baseline.
• Comparison with Static Baselines: The RL-based approach demonstrated superior performance over a static threshold-based event-triggered baseline. The static method exhibited irregular triggering patterns and larger tracking deviations, particularly during dynamic transitions, whereas the RL policy produced structured, context-aware updates that maintained geometric tracking fidelity.

Significance and Claims
The authors claim that the primary contribution of this work is the formulation of event-triggering as a learned, state-dependent decision rather than a fixed rule. This approach provides a more consistent and adaptive trade-off between accuracy and computational cost. The paper emphasizes that the framework is largely platform-agnostic, relying only on input-output data and a scalar reward, making it potentially extensible to other soft robot morphologies (e.g., pneumatically actuated or multi-segment systems) provided a persistently exciting dataset is available.

The authors modestly acknowledge limitations, noting that the "sim-to-real" gap (due to unmodeled dynamics like actuation delay and cable hysteresis) necessitated retuning of the penalty parameter $\rho$ for hardware deployment. They suggest that future work could address this through domain randomization, online data updating, and dynamic-aware simulations to further bridge the gap between simulation and reality.