⚡ electrical engineering

A KT-LSTM-SAC-Based Closed-Loop Temperature Control Strategy for Medical Robot-Assisted Moxibustion

This paper proposes a KT-LSTM-SAC-based closed-loop control strategy that integrates kinematic and thermal data via an LSTM-enhanced soft actor-critic algorithm to achieve precise, safe, and stable temperature regulation in robot-assisted moxibustion by continuously adjusting the end-effector standoff distance.

Original authors: xuelian gu, ping Li, jiale dong, yurong tao, zikai liu, chengyu yuan

Published 2026-10-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: xuelian gu, ping Li, jiale dong, yurong tao, zikai liu, chengyu yuan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

For centuries, the practice of moxibustion has relied on the steady hand and experienced eye of a practitioner. In this traditional therapy, a stick of dried mugwort is lit and held near specific points on the skin to warm the underlying tissues. The goal is to reach a precise depth of heat that stimulates the body without burning the surface. However, human skin reacts quickly to heat, while the deeper tissues warm up much more slowly. This delay creates a difficult challenge: if a practitioner moves the heat source too close to fix a slow rise in deep warmth, they risk scorching the skin before the deeper target ever feels the benefit. Conversely, if they wait too long to see the deep tissue warm up, the surface might get too hot. Making this process consistent and safe for every patient requires a level of precision that human reaction times struggle to maintain, especially when a patient breathes or shifts slightly during treatment.

To address this gap between human limitation and therapeutic need, researchers at the University of Shanghai for Science and Technology have developed a new way for robots to manage this delicate thermal balance. They created a computer simulation of a robot arm holding a heat source over a model of a human back. Instead of simply reacting to the temperature of the skin, the robot was taught to predict what is happening deep inside the tissue. The team built a system that acts like a dual-stream brain: one part watches the robot's movements and the distance between the heat source and the skin, while the other part remembers the history of temperature changes over time. By combining these two streams of information, the robot learns to adjust its position continuously, moving the heat source closer or farther away to keep the deep tissue at a steady, therapeutic warmth while keeping the skin safe.

The core of this system is a method called KT-LSTM-SAC, a name that describes how the robot thinks. The "KT" stands for kinematic-thermal, meaning the robot considers both its physical position and the heat it is generating. The "LSTM" part allows the robot to remember a sequence of past temperatures, understanding that heat takes time to travel from the skin to the deeper layers. The "SAC" component is a learning algorithm that helps the robot figure out the best actions to take through trial and error, much like how a person learns to ride a bicycle by falling and adjusting until they find their balance. In this specific study, the robot's only job was to control the distance between the heat source and the skin, moving it along a straight line perpendicular to the body. If the deep tissue was too cold, the robot moved the heat closer; if it was getting too hot, the robot moved away. Crucially, the system was trained with strict safety rules that penalized the robot if the skin temperature got too high or if the heat source got too close, forcing it to find a safe path to the target temperature.

The researchers tested this system in a highly detailed virtual environment that mimics the physics of a human body, including the way heat spreads through tissue and the way a person's chest moves with breathing. They set a goal for the deep tissue to reach and stay at 43.0 degrees Celsius, a temperature known to be effective for therapy but requiring careful management to avoid injury. They compared their new robot brain against several older methods, including standard industrial controllers and other artificial intelligence approaches that did not have the same memory of past temperatures or the same dual-stream design. The results showed that the new method was significantly better at keeping the deep tissue within the desired temperature range. In the most precise test, where the target was allowed to vary by only half a degree, the robot kept the temperature in the correct zone 94.80% of the time. It also maintained a very small average error of just 0.32 degrees Celsius.

Perhaps more importantly, the new system proved to be much safer than the alternatives. When the researchers removed the safety rules from the robot's training, it became very good at hitting the target temperature quickly, but it did so by getting dangerously close to the skin, causing the surface temperature to spike. The full system, with its safety constraints, learned to be more patient. It achieved a slightly higher average error in the deep tissue but kept the skin temperature much lower and avoided risky movements. This trade-off suggests that the robot learned to prioritize the long-term safety of the patient over the speed of heating. The study also found that the robot remained effective even when the simulation was tweaked to mimic different body types or slightly different heat properties, showing that the learning was robust and not limited to the specific numbers used in the test.

While these results are promising, the researchers are clear that this work is currently a simulation. The robot has not yet been tested on real human skin or on a physical robot arm in a hospital setting. The study demonstrates that a computer model can learn to manage the complex, delayed relationship between surface heat and deep tissue warmth better than current standard methods. It suggests that in the future, robots could assist in moxibustion by providing a level of steady, safe control that is difficult for humans to maintain consistently. The next steps for this technology will involve building physical prototypes and testing them with materials that mimic human tissue to see if the virtual success translates to the real world. Until then, the study stands as a proof of concept that artificial intelligence can learn to handle the subtle, time-delayed physics of heat therapy with both precision and care.

Technical Summary: A KT-LSTM-SAC-Based Closed-Loop Temperature Control Strategy for Medical Robot-Assisted Moxibustion

Problem Statement
Robot-assisted moxibustion currently relies heavily on operator experience and faces significant challenges regarding delayed thermal responses and insufficient closed-loop temperature stability. While existing robotic systems have advanced in acupoint localization and surface temperature feedback, they often fail to adequately account for the thermal state at the target deep-tissue depth. Deep-tissue temperature is difficult to measure noninvasively, and surface temperature alone is an insufficient proxy due to heat-transfer delays, spatial patterns, and the cumulative nature of thermal effects. Furthermore, respiratory motion introduces periodic displacements that complicate the maintenance of a consistent treatment distance. The core problem addressed is how to achieve stable, safe, and precise control of deep-tissue temperature (targeted at 43°C) using only surface observations and kinematic data, while managing the inherent delays and safety constraints of thermal therapy.

Methodology
The study proposes a Kinematic–Thermal Dual-Stream LSTM-Enhanced Soft Actor–Critic (KT-LSTM-SAC) method. The control task is formulated as the continuous adjustment of the end-effector's standoff distance along the acupoint surface normal. The methodology consists of three primary components:

Equivalent Temperature-Feedback Model:
A reduced-order model estimates the deep-tissue temperature ( $\hat{T}_{d,t}$ ) using surface temperature ( $T_{s,t}$ ), standoff distance, and a thermal-memory state. The model incorporates:
- Kinematic State: Includes the target acupoint position, surface normal, commanded and actual standoff distances, respiratory displacement (simulated via a cosine-power function), and joint states.
- Thermal State: Models equivalent heat input as an exponentially decaying function of distance. It introduces a thermal-memory state ( $m_t$ ) to capture delayed and cumulative heating effects. The deep-tissue temperature is updated based on surface temperature, a coupling coefficient, and recovery terms, simulating the slower response of deep tissue compared to the surface.
KT-LSTM-SAC Control Policy:
The control architecture utilizes a dual-stream network fed into a Soft Actor–Critic (SAC) framework:
- Kinematic Stream: A Multi-Layer Perceptron (MLP) processes motion-related variables (position, distance, respiratory disturbance).
- Thermal Stream: A Long Short-Term Memory (LSTM) network encodes the history of temperature responses (surface and estimated deep-tissue temperatures) over the preceding 50 control steps to capture temporal dependencies and delayed thermal dynamics.
- Fusion: A gated-fusion module combines features from both streams to output a continuous action ( $\Delta d_t$ ) representing the standoff-distance adjustment.
Reward Function and Safety Constraints:
The reward function balances temperature tracking with safety. It includes terms for:
- Tracking: Encouraging the estimated deep-tissue temperature to approach the target (43°C).
- Stability: Rewarding maintenance within a target band.
- Safety Penalties: Explicitly penalizing excessive surface temperature, excessive deep-tissue temperature, insufficient standoff distance, and cumulative thermal damage.
- Smoothness: Penalizing abrupt changes in the end-effector action.

Key Contributions

Dual-Stream Architecture: The paper introduces a novel KT-LSTM-SAC architecture that fuses kinematic state (including respiratory disturbance) with temporal thermal features. This allows the policy to learn the relationship between distance adjustments and delayed thermal responses more effectively than standard SAC or LSTM-SAC approaches.
Deep-Tissue Estimation for Control: Unlike previous works that rely primarily on surface temperature, this study integrates an equivalent heat-transfer model to estimate deep-tissue temperature, using this estimate as the primary feedback variable for the control loop.
Safety-Constrained Learning: The method incorporates thermal safety constraints (surface limits, cumulative damage, standoff distance) directly into the reward function, ensuring the policy learns a balance between heating efficiency and patient safety.
Comprehensive Simulation Framework: A MuJoCo-based simulation environment was developed, featuring a 6-DOF robotic arm, a human back model, respiratory motion simulation, and a dynamic thermal feedback module to rigorously test the control strategy.

Results
The proposed method was evaluated in a simulation environment at a target temperature of 43.0 ± 0.5°C and compared against PID, TD3, standard SAC, and LSTM-SAC.

Performance Metrics: KT-LSTM-SAC achieved a target-band occupancy rate of 94.80%, a Mean Absolute Error (MAE) of 0.32°C, and a mean safety cost of 3.60 × 10⁻².
Comparative Analysis:
- Compared to PID, KT-LSTM-SAC showed significantly higher success rates and lower tracking errors, demonstrating the limitations of conventional control under delayed thermal dynamics.
- Compared to standard SAC and TD3, the inclusion of LSTM for temporal feature extraction improved both tracking accuracy and safety.
- Compared to LSTM-SAC, the addition of the kinematic stream (KT-LSTM-SAC) further increased the target-band occupancy rate and reduced the mean safety cost.
- Ablation Study: Removing safety constraints (KT-LSTM-SAC w/o Safety) resulted in a lower MAE (0.29°C) but a significantly higher mean safety cost (8.36 × 10⁻²) and maximum surface temperature, confirming that the safety constraints are essential for preventing aggressive, risky heating actions.
Robustness: The method maintained 100% success rates under ±10% perturbations of key thermal model parameters, though performance degraded slightly under ±20% perturbations.
Adaptability: The controller successfully maintained performance across adjacent target temperatures (42.0°C, 42.5°C, and 43.0°C).

Significance and Claims
The paper claims that the KT-LSTM-SAC method provides a superior balance between temperature-tracking performance and thermal safety compared to existing control strategies. By jointly modeling kinematic states and temporal thermal features, the approach effectively handles the delayed and cumulative nature of heat transfer in biological tissue. The study posits that this simulation-based framework is a feasible step toward closed-loop temperature control in medical robot-assisted moxibustion. However, the authors modestly note that the current work is simulation-based; future validation using tissue-mimicking phantoms and physical robotic platforms is required to address real-world factors such as measurement noise, localization errors, and inter-individual variability in thermal response.

Technical Summary: A KT-LSTM-SAC-Based Closed-Loop Temperature Control Strategy for Medical Robot-Assisted Moxibustion

More like this