🤖 machine learning

Hierarchical Spatio-Temporal Transformer for Coherent Emergency Department Forecasting

The paper introduces HierSTT, a hierarchical Transformer-based framework that jointly forecasts emergency department demand across hospital, regional, and national levels to ensure prediction coherence, demonstrating significant accuracy improvements over existing non-hierarchical and reconciliation methods on a new nationwide Portuguese dataset.

Original authors: Filipa Lino, Bárbara Tavares, Carlos Santiago, Cláudia Soares, Manuel Marques

Published 2026-07-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Filipa Lino, Bárbara Tavares, Carlos Santiago, Cláudia Soares, Manuel Marques

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather. If you only look at the temperature in your own backyard, you might guess it will rain tomorrow. But if you also look at the weather patterns across the whole country, you realize that a massive storm front is moving in, which means your backyard will get wet, and so will the next town over. In the world of data science, this is the difference between looking at a single data point in isolation versus understanding how all the points fit together in a big, connected family. This paper lives in the corner of science called "Time Series Forecasting," which is basically the art of using past numbers to guess what will happen next. It builds on two big ideas: "Hierarchical Time Series," which is just a fancy way of saying data is organized in levels (like a single store, a whole city, and the entire country), and "Transformers," which are super-smart computer brains originally designed to read language but are now great at spotting patterns in numbers. Why do we care? Because if we get these predictions wrong, hospitals might run out of beds or staff, leaving people waiting too long for help. Getting the math right means saving time, money, and maybe even lives.

Now, let's talk about the problem the authors, Filipa Lino and her team, are tackling. Emergency Departments (EDs) are like the front door of a hospital, and they are always crowded. Predicting how many people will walk through that door is a nightmare because it depends on so many things: the day of the week, the weather, flu season, and even if a local sports team won a big game. Usually, hospitals try to predict their own busy days, and regional health bosses try to predict their whole area's needs, and national leaders try to guess the country's total. But here's the catch: they often do this separately. It's like if your family tried to plan a dinner party where everyone guessed how much food they needed individually, and then the mom tried to add it all up. The result? A chaotic mess where the total amount of food doesn't match the sum of what everyone thought they needed. The authors call this "incoherent" forecasting.

To fix this, the team created a new computer model called HierSTT (Hierarchical Spatio-Temporal Transformer). Think of HierSTT as a super-organized family dinner planner. Instead of letting everyone guess in isolation, this model acts like a wise head of the household who looks at the big picture first. It starts by predicting the demand for the entire country (the "National Level"). Then, it uses that big picture to guide the predictions for the five big regions (the "Regional Level"). Finally, it uses the regional plans to help predict the needs of each of the 81 individual hospitals. It's a "top-down" approach where the big forecast whispers instructions to the smaller ones, ensuring that if the national model says 10,000 people will visit, the sum of all the hospital predictions actually adds up to 10,000.

The model is built like a high-tech relay race. At the top, it uses a "Temporal Fusion Transformer" to understand the national trends, looking at things like holidays and mortality rates. Then, it passes the baton to "Spatio-Temporal Transformers" for the regions and hospitals. These lower-level models are special because they can talk to each other; they know that if Hospital A is busy, Hospital B nearby might be too. They also use a special "coherence-aware loss" during their training. Imagine a teacher grading a student's math homework not just on whether the final answer is right, but also on whether the steps add up correctly. If the hospital predictions don't add up to the regional total, the model gets a "penalty" and learns to fix it.

The team tested this on a real dataset from Portugal, covering 81 hospitals across 5 regions from January 2021 to April 2024. They compared their new model against old-school methods (like simple averages) and other fancy AI models that didn't talk to each other. The results were impressive. HierSTT reduced the average prediction error by about 32% compared to the best non-hierarchical deep learning model. It didn't just get the numbers right; it also kept the family math consistent. While other models might have been great at guessing the national total but terrible at guessing individual hospitals (or vice versa), HierSTT was consistently good at everything. It proved that by teaching the AI to respect the family hierarchy and the connections between different levels, you get a much clearer, more reliable picture of what's coming next. The authors suggest that this approach is a significant step forward, showing that we don't have to choose between accuracy and consistency; we can have both.

Technical Summary: Hierarchical Spatio-Temporal Transformer for Coherent Emergency Department Forecasting

1. Problem Definition

Emergency Departments (EDs) face persistent pressure from unpredictable patient demand, seasonal surges, and non-urgent visits. Effective planning requires accurate forecasts at multiple decision-making levels: hospitals need local estimates for staffing, regions need forecasts to coordinate units, and national authorities require system-wide projections.

Current approaches typically forecast ED demand in isolation at a single level (e.g., a single hospital or region), ignoring the inherent hierarchical structure where hospitals aggregate to regions, and regions aggregate to the national system. This isolation leads to incoherent predictions, where the sum of hospital-level forecasts does not match regional or national forecasts. Furthermore, data availability is heterogeneous across levels: hospital data captures operational details, while regional and national levels provide broader environmental and population signals. Existing deep learning models often fail to jointly model these hierarchical dependencies and heterogeneous covariates.

2. Methodology: HierSTT

The authors propose HierSTT (Hierarchical Spatio-Temporal Transformer), a deep learning framework designed to jointly predict ED demand across hospital, regional, and national levels while ensuring hierarchical coherence.

Architecture

The model employs a top-down architecture where higher-level representations guide lower-level forecasts:

National Level: Modeled using a Temporal Fusion Transformer (TFT). This component captures global temporal patterns and heterogeneous covariates (e.g., mortality rates, national demand) using an LSTM-based encoder-decoder with multi-head self-attention.
Regional and Hospital Levels: Modeled using Spatio-Temporal Transformer encoder-decoder modules. These components handle multiple interacting entities (5 regions, 81 hospitals).
- Encoder: Alternates between temporal self-attention (within each entity over the 42-day history) and spatial self-attention (across entities at each time step) to learn intra-entity patterns and inter-entity correlations.
- Decoder: Generates predictions for the 28-day horizon. Crucially, it incorporates higher-level forecasts (national forecasts condition regional decoding; regional forecasts condition hospital decoding) as additional inputs via cross-attention mechanisms. This allows lower-level estimates to be conditioned on global signals.

Training Objective

To enforce consistency without hard-constraining outputs, the authors introduce a coherence-aware loss function:
$\mathcal{L} = (1 - \alpha)(\mathcal{L}^{(n)} + \mathcal{L}^{(r)} + \mathcal{L}^{(h)}) + \alpha \mathcal{L}_{coh}$

$\mathcal{L}^{(\cdot)}$ represents the direct forecasting loss (Smooth L1) for each level.
$\mathcal{L}_{coh}$ is a coherence loss that penalizes inconsistencies between aggregated lower-level predictions and upper-level forecasts (e.g., sum of hospital predictions vs. regional prediction).
$\alpha$ balances accuracy and coherence.

Dataset

The work introduces a new nationwide Portuguese ED dataset covering:

81 hospitals grouped into 5 Regional Health Administrations (RHAs).
Time span: January 1, 2021, to April 20, 2024 (excluding early 2021 to mitigate COVID-19 anomalies).
Heterogeneous Covariates:
- Hospital: Triage levels, waiting times, access pathways, open/closed status.
- Regional: Aggregated demand, air quality (AQI), temperature.
- National: System-wide demand, mortality rates.

3. Key Contributions

Hierarchical Framework: A Transformer-based framework that simultaneously forecasts demand across three aggregation levels (hospital, regional, national) in a single end-to-end model.
Coherence-Aware Training: A novel loss function that enforces cross-level consistency during training, moving beyond post-hoc reconciliation methods.
New Dataset: A comprehensive, real-world dataset of Portuguese ED activity with heterogeneous covariates specific to each administrative level.

4. Experimental Results

Experiments were conducted on a 28-day forecasting horizon using 42 days of historical data. HierSTT was compared against statistical baselines (ARIMA, ETS), non-hierarchical deep learning models (LSTM, Transformer, TFT, N-BEATS), and hierarchical reconciliation methods (Top-Down, Bottom-Up, Middle-Out).

Forecasting Accuracy:
- HierSTT reduced the average Weighted Absolute Percentage Error (WAPE) by 32% relative to the best non-hierarchical deep learning baseline.
- At the regional level, it reduced MAE by ~38% and RMSE by ~27% compared to the best non-hierarchical baseline.
- At the hospital level, it improved MAE by ~8% over N-BEATS.
- At the national level, it achieved performance comparable to the standalone TFT, demonstrating that joint training does not degrade higher-level accuracy.
Hierarchical Coherence:
- HierSTT achieved the lowest Hierarchical Aggregation Error (HAgE) among all non-reconciliation models, improving prediction-side coherence by ~13% over ARIMA.
- Unlike reconciliation methods that enforce coherence algebraically (often at the cost of accuracy), HierSTT learned to produce forecasts that were both accurate and near-coherent.
- It outperformed reconciliation baselines in Ground-Truth HAgE, indicating that its lower-level forecasts aggregate to match observed higher-level values more closely than methods relying on post-hoc adjustment.

5. Significance and Claims

The paper claims that HierSTT addresses the critical gap in ED forecasting where existing methods either ignore hierarchical structures or rely on post-hoc reconciliation that fails to learn cross-level dependencies during training.

Joint Optimization: By jointly modeling temporal dynamics, heterogeneous covariates, and hierarchical dependencies, HierSTT achieves a superior balance between predictive accuracy and coherence.
Operational Relevance: The framework supports multi-level decision-making, ensuring that local staffing plans (hospital level) are consistent with regional and national resource allocation strategies.
Robustness: The model demonstrates stability across random seeds and effectively handles the heterogeneity of real-world healthcare data, including operational variables like hospital closures.

The authors conclude that effective hierarchical forecasting requires integrating structural constraints directly into the learning process rather than treating them as a post-processing step, a principle validated by the performance gains of HierSTT across all evaluation metrics.