💻 computer science

Trustworthy AI for Dengue Outbreak Forecasting Using a Temporal Fusion Transformer with Explainability and Uncertainty Estimation

This study presents a trustworthy AI framework for dengue outbreak forecasting that integrates a Temporal Fusion Transformer with probabilistic uncertainty estimation, intrinsic explainability, and rule-based risk stratification to outperform baseline models and support public health decision-making.

Original authors: Anima Dahal, Bibash Basnet

Published 2026-09-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Anima Dahal, Bibash Basnet

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the humid, tropical corners of the world, a mosquito-borne virus called dengue fever spreads with alarming speed, turning neighborhoods into zones of illness and overwhelming local hospitals. The virus thrives on specific weather conditions, flourishing when temperatures rise and rains fall, making its behavior deeply tied to the climate. For decades, public health officials have tried to stay ahead of these outbreaks, but predicting exactly when and where the virus will strike has been a difficult puzzle. Traditional methods often rely on simple statistics that struggle to see complex patterns, while newer computer systems can be so complicated that no one knows how they reached their conclusions. When a forecast is wrong, or when a prediction comes without a warning about how uncertain it is, health workers cannot plan effectively. They need more than just a guess; they need a reliable guide that explains its reasoning and admits when the future is unclear.

A team of researchers has built a new kind of artificial intelligence system designed to solve this problem, creating a tool that not only predicts dengue cases but also explains why it made those predictions and how much confidence it has in them. Using data from two cities—one in Puerto Rico and one in Peru—the researchers trained their system to look at weeks of weather reports and past dengue cases to forecast what would happen in the coming month. Unlike older models that simply spit out a single number, this new system generates a range of possible outcomes, telling officials not just how many cases to expect, but how likely it is that the number will be higher or lower. It also acts as a transparent partner, highlighting which factors, such as humidity or recent case trends, were most important in shaping the forecast. This approach transforms a "black box" computer program into a trustworthy assistant that health officials can actually use to prepare for an outbreak.

The researchers tested their system against several other methods, including simple guesses based on the last week's numbers and more complex computer models that have been used for years. Their new system, which uses a sophisticated architecture capable of remembering long-term patterns while focusing on recent changes, proved to be the most accurate. It made fewer errors than the other models, correctly tracking the rise and fall of dengue cases over a four-week period. More importantly, it provided a safety net of information that the other models lacked. When the system predicted a range of possible case numbers, the actual number of sick people fell within that range more than ninety percent of the time. This means that when the system says the outbreak will be moderate, officials can be very sure that the reality will not be drastically different, allowing them to allocate resources with greater confidence.

Beyond the numbers, the system offers a window into its own thinking, a feature that is crucial for building trust in artificial intelligence. By analyzing the data, the system identified that the most recent increase in dengue cases and changes in humidity were the strongest drivers of its predictions. It also paid the most attention to the most recent weeks of data, understanding that the immediate past is often the best indicator of the near future. This transparency allows health experts to verify that the computer is using the right logic, rather than relying on hidden patterns that might be meaningless. The researchers then took these predictions and turned them into a simple risk guide. They divided the forecasts into low, moderate, and high-risk categories, attaching specific advice to each level. If the system predicts a low risk, the recommendation is routine monitoring; if it predicts a high risk, the advice shifts to mobilizing medical teams and intensifying mosquito control efforts.

The study was conducted using historical data from two specific cities, and while the results are promising, the researchers acknowledge that the system needs to be tested in more places and with different diseases to see if it works everywhere. They also suggest that future versions could include real-time data on how people move around cities or access sanitation, which might make the predictions even sharper. For now, the work demonstrates that it is possible to build an artificial intelligence that is not only accurate but also honest about its uncertainties and clear about its reasoning. By combining strong forecasting power with transparency and practical advice, this framework offers a new way for communities to prepare for the unpredictable nature of infectious diseases, turning raw data into a clear path forward for public health.

Technical Summary: Trustworthy AI for Dengue Outbreak Forecasting

Problem Statement
Accurate forecasting of dengue outbreaks is critical for public health planning, yet existing approaches often suffer from three key limitations: they rely primarily on deterministic point predictions without quantifying uncertainty; they operate as "black boxes" with limited interpretability regarding input variables; and they lack integrated decision-support mechanisms to translate predictions into actionable public health interventions. While deep learning models like LSTMs and Transformers have improved prediction accuracy, they frequently fail to provide the reliability metrics and transparency required for real-world deployment in disease surveillance.

Methodology
The study proposes a unified Trustworthy AI framework centered on the Temporal Fusion Transformer (TFT) architecture. The methodology is structured as follows:

Dataset: The framework was evaluated using the DengAI: Predicting Disease Spread dataset, comprising weekly dengue case counts and environmental variables (temperature, humidity, rainfall, vegetation indices) from San Juan, Puerto Rico, and Iquitos, Peru.
Data Pipeline: A chronological preprocessing pipeline was employed to preserve temporal integrity. Data were split into training (70%), validation (15%), and testing (15%) sets without random sampling. Feature engineering included the extraction of calendar-based seasonality (sine/cosine transformations), lag features (1, 2, 4, and 8 weeks), and rolling statistics (mean, std, max, min) to capture historical patterns.
Model Architecture: The TFT model utilized a 24-week encoder window to predict a 4-week horizon. Key architectural components included:
- Variable Selection Networks: To identify relevant input features.
- Gating Mechanisms: To filter irrelevant information.
- Multi-head Attention: To capture long-term dependencies and interactions between static (city) and dynamic (weather, case counts) inputs.
- Probabilistic Output: The model was trained using Quantile Loss to predict the 10th, 50th, and 90th percentiles, enabling the generation of prediction intervals.
Evaluation Metrics:
- Accuracy: Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Coefficient of Determination ( $R^2$ ).
- Uncertainty: Prediction Interval Coverage Probability (PICP) and Prediction Interval Normalized Average Width (PINAW).
- Baselines: The TFT was compared against a simple baseline, a seasonal baseline, and standard LSTM models.
Decision Support: A rule-based risk stratification layer was developed to map forecasted case counts (derived from the 33rd and 66th percentiles of training data) into low, moderate, and high-risk categories, accompanied by specific public health recommendations.

Key Contributions

Unified Framework: Integration of multi-horizon forecasting, uncertainty quantification, intrinsic explainability, and risk-based decision support into a single pipeline.
Probabilistic Forecasting: Implementation of quantile-based forecasting to provide reliable confidence intervals (Q10–Q90) alongside point estimates, addressing the lack of uncertainty quantification in many existing models.
Intrinsic Explainability: Utilization of the TFT's built-in variable selection and temporal attention mechanisms to interpret feature importance and temporal dependencies without external post-hoc tools.
Actionable Output: Development of an automated module that converts probabilistic forecasts into specific risk levels and public health action plans.

Results

Forecasting Performance: The proposed TFT outperformed all baselines. It achieved an MAE of 6.560, RMSE of 10.724, and $R^2$ of 0.840. This represented a 13.70% reduction in MAE and a 17.18% reduction in RMSE compared to the simple baseline, and significant improvements over the LSTM baseline.
Uncertainty Estimation: The Q10–Q90 prediction interval achieved a PICP of 93.12% (covering 555 of 596 test samples) with a PINAW of 0.1623, indicating that the intervals were both highly reliable and relatively narrow.
Multi-Horizon Consistency: The model maintained strong performance across the 4-week forecast horizon, with $R^2$ values remaining above 0.819 even at the 4-week mark.
Explainability: Feature importance analysis identified case_increase (11.23%) as the most influential predictor, followed by humidity and temperature variables. Temporal attention weights confirmed the model prioritized recent observations within the 24-week window.
Risk Stratification: The system successfully classified test weeks into risk categories (38.8% low, 42.3% moderate, 19.0% high), providing a basis for tailored surveillance and resource allocation.

Significance and Claims
The paper claims that the primary significance of this work lies in shifting the paradigm from pure prediction accuracy to trustworthiness. By combining high-performance forecasting with uncertainty quantification and interpretability, the framework addresses the "black box" nature of deep learning and the lack of actionable guidance in current systems. The authors assert that this approach provides a more complete tool for disease surveillance, offering not just a forecast but also an assessment of its reliability and specific recommendations for public health intervention.

The study acknowledges limitations, noting that the validation was restricted to two cities within the DengAI dataset. Consequently, the authors suggest that future work should focus on validating the framework across additional locations and disease contexts, incorporating real-time data streams, and refining uncertainty calibration techniques before practical deployment in real-world public health settings.

Technical Summary: Trustworthy AI for Dengue Outbreak Forecasting

More like this