📊 statistics

General Value Functions for Remaining Useful Life and Failure-Mode Prediction

This paper proposes formulating remaining useful life and failure-mode prediction as vector General Value Function estimation using a multi-step temporal-difference estimator, which leverages partial degradation data through temporal recursion to outperform traditional supervised learning methods, particularly in scenarios with scarce complete run-to-failure labels.

Original authors: Hao Yan, Ali Sarabi, Qing Zou, Boyang Xu

Published 2026-07-27

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Hao Yan, Ali Sarabi, Qing Zou, Boyang Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a race car driver who has been driving for hours. You can see the engine temperature rising, the oil pressure dropping, and the tires wearing down. You want to know two things: how much longer the car can keep going before it breaks, and exactly how it will break—will the engine seize, will a tire blow out, or will the brakes fail? This is the heart of "predictive maintenance," a field where engineers try to peek into the future of machines using streams of data like vibration, heat, and pressure.

To do this, scientists often treat the machine's life like a series of snapshots. They look at a window of time, say the last ten minutes of sensor data, and ask a computer to guess the remaining life. But this approach has a blind spot: it treats every snapshot as a separate, isolated puzzle piece. It forgets that the car's condition at minute 10 is directly linked to minute 9, which is linked to minute 8. It's like trying to understand a movie by looking at single, unconnected frames without realizing the story flows from one to the next. The paper you are about to read tackles this problem by teaching computers to see the "flow" of time, using a mathematical idea called a "General Value Function" to connect the dots between the present moment and the inevitable end.

The Story of the Time-Traveling Crystal Ball

In the world of machines, everything eventually wears out. Whether it's a jet engine, a wind turbine, or a robot arm, they all follow a path of "degradation" until they hit a wall called "failure." The big question for engineers is: How much life is left? (This is called Remaining Useful Life, or RUL) and How will it die? (This is the Failure Mode).

Usually, to teach a computer to answer these questions, we show it thousands of examples of machines that ran all the way until they broke. We say, "Look at this data from 10 minutes before the crash; the answer is 10 minutes." This is like showing a student a finished math test and asking them to memorize the answers. It works great if you have a library full of finished tests. But what if you only have a few finished tests, and a mountain of half-finished ones? Or what if you have a pile of sensor data from anonymous machines where you don't even know which machine is which? Traditional methods struggle here because they can't make sense of the "incomplete" stories.

This paper introduces a smarter way to learn, using a concept borrowed from video game AI called General Value Functions (GVFs). Think of a GVF not as a crystal ball that predicts the future all at once, but as a chain of dominoes.

In the old way (called "Monte Carlo" learning), the computer waits until the very end of the game to see who won, then it tries to work backward to guess what happened in the middle. It's like waiting for the final score of a soccer match to guess the score at halftime. It's accurate if you have the final score, but useless if the game is still being played or if you only have a clip of the first half.

The new method in this paper uses Temporal-Difference (TD) learning. Instead of waiting for the end, the computer makes a guess about the future, then checks its guess against the next step in the data. It's like a hiker looking at the trail ahead. If the hiker guesses, "I'll reach the summit in 2 hours," and then takes a step and sees the path is steeper than expected, they immediately update their guess to "Maybe 2.5 hours." They don't wait until they reach the top to realize they were wrong; they adjust their prediction at every single step. This allows the computer to learn from partial journeys. Even if a machine hasn't broken yet, or if we lost the identity of the machine, the computer can still learn the rules of the road by connecting the "now" to the "next."

The Big Experiment: Simulations and Real Engines

The authors tested this idea in two very different worlds.

1. The Video Game Simulation
First, they built a virtual world where they could control everything. They created 600 digital machines that degraded over time, splitting them into two types of failure modes (like "engine fire" vs. "gear slip").

The Full Story: When they gave the computer the complete history of every machine from start to finish, the new "domino" method (TD) performed just as well as the old "wait-for-the-end" method (MC).
The Missing Pages: Then, they made it harder. They took away the endings of many stories. They gave the computer only anonymous, short clips of machines that hadn't failed yet. Here, the old method crashed. Without the final "game over" label, it had nothing to learn from. The new TD method, however, kept learning. It used the "next step" to update its guesses, effectively filling in the missing pages of the story. In these "label-scarce" situations, the new method was significantly better at predicting how much time was left. However, for predicting how the machine would fail, the simulation showed that when data was extremely scarce (only 10-50% of labels available), the computer's ability to guess the failure mode was still near random chance, just like the old methods. The real breakthrough was in the time prediction, not the failure type prediction, under those specific conditions.

2. The Real-World Test: NASA's Jet Engines
Next, they moved to real data from NASA's C-MAPSS dataset, which contains recordings of turbofan jet engines. These engines are complex, running under different conditions and failing in different ways.

The Challenge: The real data is messy. Sometimes the engines are cut off before they fail (censored data), and the "failure modes" aren't clearly labeled in the original data.
The Result: The new method again showed its strength. On the hardest datasets (where engines have multiple ways to fail and operate under different conditions), the TD method predicted the remaining life more accurately than the traditional methods. It was particularly good at the "stitch" setting, where the training data was fragmented and anonymous. Even with only 10% to 50% of the usual labeled data, the TD method managed to predict the remaining life with high accuracy. While it did show some improvement in predicting the failure mode compared to the old methods in these fragmented settings, the primary and most consistent victory was in accurately forecasting how much time was left, rather than perfectly identifying the specific failure type in every case.

What This Means for the Future

The paper doesn't claim to have solved the mystery of machine failure forever. It suggests that by changing how we ask the computer to learn—shifting from "memorize the final answer" to "update your guess at every step"—we can make predictive maintenance much more efficient.

The key takeaway is that fragmented data is not useless data. In the past, if a machine's record was incomplete or if we didn't know its identity, engineers often threw that data away. This paper shows that we can use those broken, anonymous pieces to teach the computer the rules of degradation, as long as we let the computer connect the dots between one moment and the next.

By treating the machine's life as a continuous story rather than a stack of disconnected snapshots, we can build systems that are smarter, require less perfect data, and can warn us about failures even when the story isn't finished yet. It's a small shift in perspective, but for the machines keeping our world running, it could mean the difference between a scheduled repair and a sudden, costly breakdown.

Technical Summary: General Value Functions for Remaining Useful Life and Failure-Mode Prediction

Problem Formulation
Predictive maintenance relies on forecasting Remaining Useful Life (RUL) and classifying failure modes from sensor trajectories. Traditional data-driven approaches typically employ fixed-window supervised learning, treating successive time windows as independent training examples with complete terminal labels (run-to-failure). This paradigm fails to naturally encode the temporal recursion linking successive degradation states, particularly when observations are partial, censored, or lack unit identity. Furthermore, standard methods often treat RUL (regression) and failure-mode classification as separate tasks, ignoring their shared origin in the same future terminal event.

The authors formalize prognostics as a vector General Value Function (GVF) prediction problem on an absorbing degradation process. In this framework:

The system evolves through a predictive state space $S$ until reaching an absorbing failure set $F$ .
RUL is defined as the expected time (hitting time) to reach $F$ .
Failure-mode prediction is defined as the probability of absorption into specific subsets $F_k \subset F$ .
These targets are not independent labels but temporally extended predictions governed by Bellman consistency: the prediction at the current state must align with the immediate transition and the prediction at the successor state.

Methodology
The paper proposes a multi-step Temporal-Difference (TD) estimator, TD(n, $\lambda$ ), to learn vector GVFs from partial trajectories without requiring complete terminal labels for every training segment.

Vector GVF Definition: The method unifies RUL and failure-mode prediction into a single vector $V(s)$ $V (s)$ .
- The RUL component accumulates operating time (cumulant = 1) until absorption.
- Failure-mode components accumulate indicators of specific terminal events.
- Continuation functions ( $\gamma$ ) control the effective horizon, allowing for soft-horizon predictions or undiscounted hitting probabilities.
Bellman Consistency: The targets satisfy the vector Bellman equation $V(s) = E[u_{t+1} + \Gamma_{t+1}V(S_{t+1}) | S_t = s]$ . This enforces temporal coherence directly at the target level, rather than relying on architectural constraints or auxiliary penalties.
TD(n, $\lambda$ ) Estimation: Instead of waiting for the full return (Monte Carlo), the algorithm bootstraps from successor predictions.
- TD(n): Uses observed transitions for $n$ steps and bootstraps the remainder using the current value estimate.
- Truncation ( $\lambda$ ): Combines $k$ -step TD targets for $k=1$ to $n$ to balance bias and variance.
- Handling Censoring: Crucially, TD targets remain well-defined for non-terminal segments (censored data) because they rely on local transitions and bootstrapped successors, whereas Monte Carlo targets require the terminal event to be observed.
Architecture: A shared convolutional neural network (CNN1D) encoder processes sensor windows, feeding into two heads: one for RUL and one for failure-mode probabilities. The network is trained jointly using a weighted Mean Squared Error (MSE) loss on the vector TD targets.

Key Contributions

Unified Formulation: The paper reframes dynamic prognostics as vector-GVF prediction on an absorbing process, unifying RUL and failure-mode prediction as temporally extended hitting-time and hitting-probability questions.
Theoretical Analysis:
- Identifies the Bellman fixed point targeted by vector GVFs.
- Characterizes the linear projected-TD limit and its relation to complete-return Monte Carlo regression under realizability.
- Provides a variance decomposition showing that, under specific conditions, bootstrapped TD targets can exhibit lower variance than complete Monte Carlo returns, particularly when terminal labels are scarce or the episode length is long.
Algorithmic Development: Develops a TD(n, $\lambda$ ) estimator capable of learning from fragmented, identity-free degradation records by utilizing local Bellman transitions instead of discarding data until complete run-to-failure labels are available.

Experimental Results
The method was evaluated on an event-triggered multimode simulation and the NASA C-MAPSS benchmark (specifically subsets FD003 and FD004 with competing failure modes).

Full-Trajectory Setting: When complete run-to-failure labels are available, the TD method achieves RUL accuracy comparable to a supervised same-backbone Monte Carlo (MC) control. However, TD consistently outperforms MC in failure-mode classification (Macro-F1) across all life stages, with the largest gains occurring early in the unit's life.
Label-Scarce Stitch Setting: In scenarios where training data consists of anonymous, partial segments (without unit identity or terminal labels for non-terminal segments):
- TD remains competitive and robust.
- MC and Health Index (HI) baselines degrade significantly. MC cannot form targets for segments that do not reach failure, and HI suffers from calibration issues under anonymity.
- On C-MAPSS, TD trained on 50% of the data (stitch) achieved RUL error rates comparable to MC trained on full trajectories.
Failure-Mode Prediction: TD demonstrated superior stability and higher mean F1 scores in the label-scarce setting, effectively propagating mode-hitting targets from terminal anchors to non-terminal windows via Bellman recursion.

Significance and Claims
The paper claims that viewing prognostics through the lens of General Value Functions and Temporal-Difference learning offers a principled way to handle the temporal structure of degradation. By treating RUL and failure modes as temporally consistent targets rather than independent window-level labels, the method enables:

Data Efficiency: Utilizing fragmented and anonymous degradation records that would otherwise be discarded in supervised pipelines.
Temporal Coherence: Enforcing consistency between successive predictions via the Bellman equation, which is intrinsic to the target definition rather than an external penalty.
Robustness: Maintaining performance in failure-mode classification and RUL estimation even when complete terminal labels are scarce or unavailable for a significant portion of the training data.

The authors position this work as a shift from indirect health-index modeling or direct supervised regression toward a framework that explicitly models the forward-looking, recursive nature of degradation processes.

The Story of the Time-Traveling Crystal Ball

The Big Experiment: Simulations and Real Engines

What This Means for the Future

Technical Summary: General Value Functions for Remaining Useful Life and Failure-Mode Prediction

More like this