🤖 AI

PEFT-MuTS: A Multivariate Parameter-Efficient Fine-Tuning Framework for Remaining Useful Life Prediction based on Cross-domain Time Series Representation Model

This paper introduces PEFT-MuTS, a parameter-efficient fine-tuning framework that leverages cross-domain pre-trained time-series models and a novel multivariate fusion mechanism to achieve accurate remaining useful life prediction with less than 1% of target equipment data, effectively overcoming the limitations of data scarcity in few-shot scenarios.

Original authors: En Fu, Yanyan Hu, Zengwang Jin, Kaixiang Peng

Published 2026-08-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: En Fu, Yanyan Hu, Zengwang Jin, Kaixiang Peng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Every machine, from the jet engine powering an airliner to the bearings inside a factory robot, eventually wears out. Predicting exactly when a piece of equipment will fail is the holy grail of maintenance, allowing engineers to fix things just before they break rather than waiting for a disaster or replacing parts too early. This prediction, known as the "remaining useful life," relies on listening to the machine's sensors as it ages. For decades, the most accurate way to teach computers to make these predictions has been to feed them massive amounts of data showing a machine's entire life, from its first day of operation until it finally stops working. However, in the real world, collecting such complete histories is often impossible. Many machines run for decades, making a full lifecycle record a project that takes a lifetime. Others are custom-built or brand new, meaning there is no historical data at all to study. When data is scarce, the smartest computer models usually fail, leaving engineers to guess.

A team of researchers has developed a new approach that solves this problem by changing how the computer learns in the first place. Instead of trying to teach a model specifically about how a jet engine or a bearing wears down, they first teach it a much broader lesson: how time itself behaves. They trained a computer model on a vast collection of unrelated time-based data, such as human brain waves recorded during sleep. This model learned to recognize universal patterns in how things change over time—trends, rhythms, and sudden shifts—without ever seeing a piece of machinery. Once the model understood these general patterns of time, the researchers showed it a tiny handful of examples from the specific machine they wanted to monitor. Because the model already understood the language of time, it could instantly recognize the signs of wear in the new machine, even with less than one percent of the data usually required.

The researchers call their method PEFT-MuTS, a system designed to be efficient and adaptable. The core idea is to take a powerful "backbone" model that has already learned from millions of unrelated time records and gently adjust it for a specific job. In the past, trying to adapt such a large model to a new task with very few examples would be like trying to steer a massive ship with a tiny rudder; the model would either get confused and forget what it learned, or it would memorize the few examples too perfectly and fail to predict the future. To prevent this, the researchers built a special, lightweight adapter that sits between the big model and the final prediction. This adapter allows the model to process each sensor on the machine independently, preserving the careful lessons it learned about time, while also having a dedicated channel to combine information from all the different sensors. This combination allows the system to see the whole picture of the machine's health without getting overwhelmed by the lack of data.

A critical discovery in this work was that the way the final prediction is set up matters just as much as the data itself. When the researchers started the training process with a standard setup, the results were unstable, swinging wildly from one prediction to the next. They found that by starting the final prediction layer with a value of zero, they could stabilize the entire process. This simple adjustment acted like a calm anchor, allowing the model to learn steadily from the few examples available without getting thrown off by the noise. The result is a system that is remarkably stable and accurate, even when it has never seen the specific machine before and has only a few snapshots of its current condition to work with.

The team tested this approach on two very different types of equipment: a dataset of aircraft engines and a dataset of industrial bearings. In both cases, they simulated a scenario where they had almost no training data, representing a situation where a machine is new or has a very short history. The new method consistently outperformed existing techniques that rely on finding similar machines or using complex meta-learning strategies. While other methods struggled to make sense of the limited data, often guessing that the machine was either brand new or already broken, the new system correctly identified the middle stages of wear. It managed to predict the remaining life of the equipment with high accuracy using only a fraction of the data that traditional methods require.

This work suggests a fundamental shift in how we approach machine learning for industrial safety. For years, the focus has been on finding machines that are similar to the one being monitored, hoping that their wear patterns match. This new approach argues that the most useful knowledge is not about specific machines, but about the universal behavior of time and change. By learning from a wide variety of unrelated time series, a model can become a general expert on degradation, ready to be applied to any new machine with minimal effort. The findings indicate that we do not need to wait for decades of data to build reliable maintenance systems; we can instead build models that understand the rhythm of time and apply that understanding to the machines of tomorrow, ensuring they run safely and efficiently even when their history is just beginning.

Technical Summary: PEFT-MuTS for Few-Shot RUL Prediction

Problem Statement
Data-driven Remaining Useful Life (RUL) prediction is currently constrained by the scarcity of full-lifecycle degradation data in real-world industrial settings. Collecting run-to-failure data is costly and often infeasible, particularly for new or customized equipment. Existing few-shot learning approaches, primarily relying on domain adaptation and meta-learning, face a critical limitation: they require sufficient degradation data from similar equipment or operating conditions to construct a source domain. When such homologous data is unavailable, these methods struggle to transfer knowledge effectively. Furthermore, adapting large pre-trained time-series models to multivariate RUL tasks presents two specific challenges: (1) how to leverage univariate pre-trained backbones for multivariate sensor data without disrupting learned temporal priors, and (2) how to design a fine-tuning strategy that ensures stable convergence with extremely limited target samples.

Methodology: PEFT-MuTS Framework
The paper proposes PEFT-MuTS (Parameter-Efficient Fine-Tuning for Multivariate Time-Series), a framework that shifts the transfer paradigm from "identifying similar degradation patterns" to "leveraging generalizable temporal priors." The framework operates in two stages:

Cross-Domain Self-Supervised Pre-training: A backbone network (specifically a 1-D ResNet-18) is pre-trained on large-scale, diverse, univariate time-series datasets (e.g., SleepEEG) using self-supervised methods like FEI or SimMTM. This stage learns universal temporal representations (trends, periodicity, frequency patterns) independent of specific equipment degradation, decoupling the transfer process from the need for equipment-specific historical data.
Parameter-Efficient Fine-Tuning: The pre-trained backbone is frozen, and a lightweight PEFT-MuTS network is inserted to adapt the model to the downstream multivariate RUL task. This network consists of three key components:
- Independent Feature Tuning Network: To preserve the univariate nature of the pre-trained backbone, this module processes each sensor variable independently using low-rank mappings (inspired by LoRA). It employs a SiLU activation to regulate information flow, ensuring the backbone's prior structure remains intact while allowing minimal parameter updates.
- Meta-Variable-Based Low-Rank Feature Fusion: To address the multivariate nature of degradation data, a dedicated learnable "meta-variable" ( $u$ ) is introduced. This variable acts as a fusion hub, dynamically aggregating information from all independent sensor variables via a low-rank space. A self-gating mechanism (Gate) filters and weights the contributions of different variables before they are fused into the meta-variable, allowing the model to capture cross-variable dependencies without disrupting the independent feature extraction of the backbone.
- Zero-Initialized Regressor: To stabilize fine-tuning under few-shot conditions, the regression head is initialized to zero. Theoretical analysis suggests that random initialization in low-data regimes leads to high gradient variance and unstable convergence. Zero-initialization minimizes initial gradient variance, preventing the model from diverging or getting trapped in poor local minima during the early stages of training.

Key Contributions

Paradigm Shift: The study demonstrates that cross-domain time-series representation models are highly effective for RUL prediction even when target degradation data is extremely scarce (<1% of samples). This broadens the scope of transferable source data beyond similar equipment to any large-scale time-series dataset.
Novel PEFT Architecture: PEFT-MuTS introduces a mechanism to adapt univariate pre-trained models to multivariate tasks. It combines independent variable tuning (to preserve pre-trained priors) with a meta-variable fusion mechanism (to learn cross-variable dependencies) using a minimal number of trainable parameters (approx. 0.50% of the backbone).
Stabilization Strategy: The paper identifies the instability of fine-tuning pre-trained models on small samples and proposes a zero-initialized regressor as a simple, effective solution to improve convergence stability and reduce prediction variance.

Experimental Results
The framework was evaluated on the C-MAPSS (aero-engine) and XJTU-SY (industrial bearing) datasets under various few-shot scenarios (ranging from 0.72% to 14.4% of original data).

Performance: PEFT-MuTS consistently outperformed conventional supervised learning methods (e.g., ResNet, DAMCNN), domain adaptation techniques (TACDA), and meta-learning approaches (MKDPINN). It achieved superior accuracy in terms of MAE, RMSE, and MAPE/SMAPE across almost all datasets.
Comparison with Baselines: While full fine-tuning of pre-trained models showed improvement over random initialization, it was prone to overfitting in few-shot settings. Linear fine-tuning was stable but lacked adaptability. PEFT-MuTS achieved the best balance, significantly outperforming DTL (a Side-Tuning baseline) by effectively utilizing multivariate information.
Ablation Studies:
- Removing cross-domain pre-training caused significant performance degradation.
- Removing the meta-variable fusion mechanism reduced performance, particularly on datasets with many variables (C-MAPSS), though the impact was smaller on low-variable datasets (Bearings).
- Removing zero-initialization led to higher variance in results and slower convergence, confirming its role in stabilizing the optimization process.
Efficiency: The method added only ~74.9K parameters (0.50% of the backbone) and maintained computational efficiency comparable to baseline models.

Significance and Claims
The paper claims that PEFT-MuTS offers a new perspective on few-shot RUL prediction by decoupling the transfer process from the requirement for equipment-specific degradation data. By leveraging general temporal priors learned from diverse cross-domain datasets, the framework enables accurate prediction even when historical data from similar equipment is unavailable. The proposed method substantially reduces the data required to achieve high predictive accuracy, making deep learning-based RUL prediction more feasible for new, customized, or data-scarce industrial assets. The authors position this work as a foundation for future research into quantifying source-domain transferability and handling more complex industrial challenges such as noisy labels and censored trajectories.

Technical Summary: PEFT-MuTS for Few-Shot RUL Prediction

More like this