📈 economics

A hybrid patch--variate attention network with reversible instance normalization, a learned market embedding, and a linearautoregressive residual for multi-horizon panel forecasting of paddy prices

This paper proposes a hybrid patch-variate attention network incorporating reversible instance normalization, learned market embeddings, and a linear autoregressive residual to achieve state-of-the-art multi-horizon forecasting of volatile Indian paddy prices, demonstrating statistically significant improvements over strong baselines specifically at decision-relevant seven and fourteen-day horizons.

Original authors: Madhusikta Tanaya Singh, Alakananda Tripathy

Published 2026-11-11✓ Author reviewed ⓘ

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Madhusikta Tanaya Singh, Alakananda Tripathy

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the bustling wholesale markets of India, known as mandis, the price of a sack of rice is never static. It swings daily, driven by the arrival of harvest trucks, the demands of festivals, and the whims of the weather. For a farmer or a trader, knowing the exact price for tomorrow is often less useful than understanding the direction the market is heading over the next week or two. If they can anticipate a rise, they might hold their produce; if they see a fall, they might sell quickly to avoid losses. However, predicting these movements is notoriously difficult. The daily price changes often look like random noise, making it easy for a computer model to appear accurate by simply guessing that tomorrow's price will be the same as today's. This trick works well for minimizing numerical errors but fails completely at the only thing that matters to a decision-maker: getting the direction right.

Researchers at Siksha 'O' Anusandhan University in India set out to solve this specific problem for Grade-A paddy, a high-quality rice variety, across eleven major markets in the state of Telangana. They faced three distinct hurdles. First, the daily price data is so noisy that standard models struggle to find a signal. Second, looking at just one market at a time provides too little data for advanced computer learning systems to work effectively without memorizing the past rather than learning from it. Third, many previous studies claimed success by focusing on small numerical improvements in error rates while ignoring whether the models could actually predict if prices would go up or down. To address these issues, the team built a new forecasting system that treats all eleven markets as a single, connected group rather than isolated islands. Instead of trying to guess the exact price number for the future, the system focuses on predicting the change in price—the difference between today and a future date. This shift in focus forces the model to learn the actual movement of the market rather than just repeating the last known value.

The researchers combined this new way of looking at the problem with a sophisticated architecture that blends several established techniques. They fed the model historical price data, information about how much rice arrived at the markets, and real-time weather data from NASA. The system uses a "panel" approach, meaning it learns the unique characteristics of each of the eleven markets while also recognizing the common forces that move them all together, such as regional weather patterns or national policy shifts. To handle the fact that rice prices have been rising over the years, the model includes a mechanism to adjust for these long-term trends, ensuring it doesn't get confused by the difference between older, cheaper data and newer, more expensive data. Crucially, the model does not rely solely on complex neural networks; it also includes a straightforward, linear path that captures the basic tendency of prices to follow their recent history. This hybrid design allows the system to use simple, reliable rules for short-term predictions while using its more complex parts to spot deeper patterns for longer-term forecasts.

When the team tested their system against a field of strong competitors—including other advanced artificial intelligence models and traditional statistical methods—they found a clear, honest picture of what works and what does not. The new model performed better than all the others at every time frame they tested, whether looking one day, one week, or two weeks ahead. However, the statistical significance of this improvement depended entirely on the time horizon. For a prediction just one day into the future, the new model was not statistically better than the best existing methods; the daily noise was simply too strong for any model to overcome with certainty. But as the forecast window widened to seven days, the new model's advantage became statistically clear. By fourteen days, the improvement was highly significant. In practical terms, this means the system is genuinely useful for the timeframes that actually matter to a farmer deciding when to sell.

The study also measured "directional accuracy," which is simply the percentage of times the model correctly guessed whether the price would rise or fall. The new model achieved an accuracy of nearly 60 percent across all timeframes, a substantial leap from the baseline of about 46 percent, which represents the score of simply guessing the most common direction every time. This result proves that the system has learned a genuine skill rather than just finding a statistical trick. Perhaps most importantly, the researchers broke down their own model to see which parts were doing the heavy lifting. They discovered that for the one-day forecast, the most valuable components were the simple linear rules and the decision to predict the change in price rather than the price itself. The more complex, attention-based parts of the system, which are often the most expensive to run, contributed very little to the short-term prediction. Their true value emerged only at the longer horizons of one and two weeks, where they helped the model navigate the complex interplay of market factors.

The researchers were careful to report their findings without exaggeration. They acknowledged that for a one-day forecast, their model is not a magic bullet and performs similarly to other top-tier methods. They also noted that the improvement in raw error numbers, while statistically significant, is small in absolute terms. The real breakthrough is the ability to reliably predict the direction of price movement over a week or two, a capability that previous models struggled to demonstrate with statistical confidence. By pooling data from multiple markets and focusing on the change in price rather than the price level, the team created a tool that is robust enough to handle the volatility of agricultural markets. The study concludes that while no model can perfectly predict the chaotic daily fluctuations of rice prices, a carefully designed system can provide a reliable edge for decision-makers looking a week or two into the future, turning a noisy, unpredictable market into one with a discernible pattern.

Technical Summary: A Hybrid Patch–Variate Attention Network for Multi-Horizon Panel Forecasting of Paddy Prices

Problem Formulation
The paper addresses the challenge of short-horizon forecasting for daily wholesale paddy prices in Indian agricultural markets (mandis). The authors identify three specific properties that make this task difficult and cause existing literature to overstate success:

Random Walk Behavior: Daily price changes are dominated by idiosyncratic noise. Models targeting the price level often minimize Root Mean Square Error (RMSE) by predicting near-persistence (the last observed value), failing to capture the direction of change, which is critical for decision-makers.
Data Scarcity: Single-market time series are too short (a few thousand observations) to train complex attention-based models without overfitting.
Weak Evaluation: Prior work often lacks adversarial baselines, ignores directional accuracy relative to a majority base rate, and fails to test if improvements exceed sampling noise using statistical significance tests.

The specific task is to forecast the price of Grade-A paddy across eleven markets in Telangana for horizons of 1, 7, and 14 days, using data from 2014 to 2025.

Methodology
The proposed solution is a hybrid deep learning architecture designed to address the specific data properties through a panel formulation and a returns-based target.

Data and Features:
- Panel Pooling: Eleven markets are modeled jointly. A learned market embedding ( $e_m$ ) captures market-specific characteristics, while a cross-sectional market factor ( $f_t$ ) is computed as the daily mean log-price across all active markets. This factor and its momentum serve as inputs to capture common shocks.
- Returns Target: Instead of predicting the price level $y_{m,t+h}$ , the model predicts the $h$ -step log-price change $\Delta^{(h)}_{m,t} = y_{m,t+h} - y_{m,t}$ . The final price is reconstructed using the known anchor $y_{m,t}$ . This shifts the learning objective to the directional signal.
- Covariates: The model incorporates endogenous features (price lags, rolling statistics, momentum, arrivals) and exogenous features (calendar variables and NASA POWER meteorological data).
Architecture:
The model is a hybrid of established components, each mapped to a specific data property:
1. Reversible Instance Normalization (RevIN): Applied to the input window to handle distribution shifts between the training period (lower prices) and the test period (higher prices).
2. Patch Branch: The time series is segmented into overlapping patches (length 12, stride 6). A self-attention mechanism operates over these patches to capture local temporal semantics (inspired by PatchTST).
3. Inverted Variate Branch: Following iTransformer, the entire series for each variable is embedded as a single token. Attention operates across variables (channels) rather than time steps to capture feature correlations.
4. STAR Module: Two centralised STAR (Star Aggregate–Redistribute) layers replace pairwise attention with a global core aggregation, providing linear complexity and robustness under distribution drift.
5. Endo/Exo Cross-Attention: Following TimeXer, endogenous (price) tokens attend to exogenous (weather/calendar) tokens, allowing external context to influence the prediction without being averaged out.
6. Linear Autoregressive Residual: A parallel explicit linear path (inspired by DLinear) maps the last observed feature vector directly to the output. The final prediction is the sum of the nonlinear head's output and this linear path.
Training and Evaluation:
- Models are trained with AdamW and a Huber loss on the scaled change.
- Evaluation uses a chronological split (70/15/15) with strict leakage controls.
- Metrics include RMSE, MAE, MAPE, $R^2$ , and Directional Accuracy (DA).
- Significance is tested using the Diebold–Mariano (DM) test comparing squared errors against strong baselines.

Key Contributions

Panel Formulation: A pooled multi-market approach with a cross-sectional factor and learned embeddings to overcome data scarcity and capture common market shocks.
Returns Formulation: A target reformulation that predicts price changes rather than levels, recovering genuine directional skill and reducing to a simple forecast when the predicted change is zero.
Hybrid Architecture: A novel assembly of current forecasting components (RevIN, Patch/Inverted Attention, STAR, TimeXer-style cross-attention, and a linear residual path) explicitly justified by data properties.
Rigorous Evaluation: A study design featuring strong baselines (DLinear, PatchTST, iTransformer, LightGBM, LSTM, Bi-LSTM) trained under identical conditions, multi-seed reporting, and the Diebold–Mariano test for statistical validation.

Results

Performance: The proposed model achieves the lowest error (RMSE) and highest $R^2$ at all horizons (1, 7, and 14 days).
Statistical Significance:
- At 1-day ahead, the improvement over the strongest baseline (Bi-LSTM) is not statistically significant ( $p = 0.17$ ).
- At 7-day ahead, the improvement becomes significant ( $p = 0.026$ ).
- At 14-day ahead, the improvement is strongly significant ( $p < 0.001$ ).
Directional Accuracy: The model achieves ~60% directional accuracy across all horizons, significantly outperforming the majority base rate of ~45.8%.
Ablation Study: Removing the linear autoregressive path causes the largest performance drop (+9.5 RMSE), indicating it carries most of the short-horizon skill. Attention-based components (Patch, STAR, Inverted branches) contribute minimally at 1-day but become more relevant at longer horizons. Weather covariates provide marginal gains for the deep model at short horizons but help weaker baselines.

Significance and Claims
The paper claims a specific and limited contribution: on pooled Telangana paddy markets, a returns-based panel model with a cross-sectional factor offers a small but statistically significant edge over modern baselines at decision-relevant horizons (1–2 weeks).

The authors explicitly state that:

The model is not significantly better than strong baselines at the 1-day horizon.
The absolute RMSE margins are small (often <1%), making the directional accuracy and statistical significance at longer horizons the primary practical findings.
The "heavier" neural machinery (attention modules) is not the primary driver of short-term gains; the returns formulation and the linear autoregressive path account for most of the skill.
The honesty regarding the lack of significance at the 1-day horizon is presented as a core contribution, avoiding the inflation of results common in the field.

The work does not claim a universal architectural victory but rather a successful application of specific design choices (panel pooling, returns target, linear residual) to a specific, noisy agricultural forecasting problem.

Technical Summary: A Hybrid Patch–Variate Attention Network for Multi-Horizon Panel Forecasting of Paddy Prices

More like this