📈 economics

Tweedie's Formula, Variance Functions, and Score-Driven Updating

This paper establishes a Bayesian interpretation of score-driven models by demonstrating that Tweedie's formula links conditional score updates to posterior mean corrections in natural exponential families, thereby unifying empirical Bayes, approximate filtering, and dynamic generalized linear models while clarifying the relationship between exact Bayesian updating and local score-based approximations.

Original authors: Peter Reinhard Hansen, Chen Tong

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Peter Reinhard Hansen, Chen Tong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the true temperature of a room, but your thermometer is a bit shaky and gives you noisy readings. You want to update your best guess every time you get a new reading.

This paper is about how to update your guesses when you are dealing with time-varying data (like stock prices, weather, or disease counts). Specifically, it connects two different ways of thinking about this problem: a "Bayesian" way (using probability and prior beliefs) and a "Score-Driven" way (using a specific mathematical rule to adjust your guess).

Here is the breakdown of the paper's main ideas using simple analogies:

1. The Two Ways of Updating

Think of updating your guess as a driver correcting their steering wheel.

The "Bayesian" Way (The Perfect Navigator): This is like having a GPS that knows the exact map of the world and the exact probability of every possible route. It calculates the perfect correction based on all the information it has. However, in the real world, this "perfect map" is often too complicated to calculate instantly. It requires solving a massive puzzle every second.
The "Score-Driven" Way (The Smart Driver): This is like a driver who doesn't have the perfect map but uses a simple rule: "If the road looks like it's curving left, turn the wheel a little bit to the left." This rule is based on the "score" (a mathematical signal) of how the current data looks. It's fast, easy to calculate, and works well in practice, but it's an approximation, not the perfect solution.

2. The Secret Connection: Tweedie's Formula

The paper's main discovery is that these two ways are actually more similar than we thought.

The authors use a famous mathematical trick called Tweedie's Formula. Imagine Tweedie's Formula is a magic decoder ring.

In a simple, perfect world (Gaussian models), this formula proves that the "Perfect Navigator's" correction is exactly the same as the "Smart Driver's" rule. The messy, complex calculation of the perfect map turns out to be just a simple adjustment based on the current noise.
The paper shows that this magic decoder ring works not just for simple temperature readings, but for a whole family of complex data types (like counts of raindrops or insurance claims), provided you adjust for the "shape" of the data.

3. The "Base Measure" Adjustment

The paper points out a subtle catch. When you use the magic decoder ring for complex data (like counting things), you have to subtract a small "background noise" factor (called the base measure) to get the right answer.

Analogy: Imagine you are trying to hear a whisper in a noisy room. The "score" tells you how loud the whisper is, but you also have to account for the hum of the air conditioner (the base measure). If you don't subtract the air conditioner's hum, your guess will be wrong.
The paper explains that while the "Perfect Navigator" needs to account for this hum, the "Smart Driver" (the score-driven model) naturally ignores it because they are looking at the problem from a different angle. This is why the two methods usually look different, but can be made to match under specific conditions.

4. The "Inverse-Fisher" Scaling (The Volume Knob)

When the "Smart Driver" turns the wheel, how hard should they turn?

The paper explains that the "score" (the signal to turn) comes with a built-in volume knob called the Variance Function. This knob automatically adjusts the signal based on how noisy the data is.
To get the perfect correction, you sometimes need to turn the volume back down using a specific setting called Inverse-Fisher scaling.
Analogy: If you are driving on a slippery road (high noise), you need to make small, gentle steering adjustments. If the road is dry (low noise), you can make bigger turns. The paper shows that the "Smart Driver" uses a specific mathematical rule to set this volume knob perfectly, so they don't overreact to noise.

5. When Do They Match Exactly?

The paper identifies a special "Goldilocks" zone where the "Smart Driver" becomes the "Perfect Navigator."

This happens when:
1. The data follows a specific mathematical family (Natural Exponential Families).
2. You start with a specific type of prior belief (Conjugate Priors).
3. You use the "Inverse-Fisher" volume knob.
4. You are looking at the "Expectation" (the average) of the data.
The Result: In this specific zone, the simple, fast "Score-Driven" rule isn't just an approximation; it is mathematically identical to the complex, perfect Bayesian calculation.

Summary

The paper argues that Score-Driven models (which are popular in finance and economics) are not just random guesses. They are actually:

Local approximations of the perfect Bayesian solution for most complex data.
Exact solutions for a specific, important class of data problems.

It gives us a new way to understand why these models work so well: they are essentially using a clever shortcut (the score) to mimic the behavior of a perfect Bayesian filter, and in many cases, they are doing it perfectly.

Technical Summary: Tweedie's Formula, Variance Functions, and Score-Driven Updating

Problem Statement
Score-driven models provide a parsimonious framework for introducing time variation into the parameters of conditional distributions by updating parameters proportionally to the score of the conditional log-likelihood. While these models are well-established for their computational tractability, robustness, and information-theoretic optimality (e.g., Blasques et al., 2015; Gorgi et al., 2024), their connection to exact Bayesian inference has remained distinct. Specifically, exact Bayesian updating in dynamic settings typically requires integrating over latent states to obtain the posterior mean, a process that is generally intractable in closed form. Score-driven models, by contrast, utilize the conditional likelihood score evaluated at the current predictive parameter. This paper addresses the gap between these two approaches: it seeks to provide a rigorous Bayesian interpretation of score-driven updating by linking it to Tweedie's formula and clarifying the conditions under which score-driven corrections coincide with exact Bayesian posterior updates.

Methodology
The paper employs a theoretical derivation approach, bridging empirical Bayes theory, natural exponential families (NEF), and approximate filtering. The methodology proceeds in three logical stages:

Revisiting Tweedie's Formula: The authors first establish the connection between Bayesian posterior corrections and marginal scores. In the Gaussian location model, Tweedie's formula (Robbins, 1956; Efron, 2011) expresses the posterior mean of a latent signal as the observed value plus a scaled score of the marginal predictive density. This is extended to natural exponential families, where the posterior mean of the natural parameter is shown to be the score of the marginal density adjusted by the base measure.
Local Bayesian-Fisher Approximation: For general conditional densities where exact marginal integration is unavailable, the paper derives a local approximation. By assuming a Gaussian predictive distribution for the latent state and applying a Fisher-scoring quadratic expansion to the log-likelihood, the authors show that the posterior correction can be approximated by an inverse-Fisher-scaled conditional score. This approximation relies on steady-state precision discounting to define the local predictive precision.
Exact Equivalence in Conjugate NEF: The authors identify a specific setting where the approximation becomes an exact algebraic identity. By considering natural exponential families with conjugate priors, steady-state precision discounting, and updates performed in the expectation parameter space, they demonstrate that the score-driven correction (using inverse-Fisher scaling) is identical to the Bayesian posterior mean before transition dynamics are applied.

Key Contributions and Results

Bayesian Interpretation of Score-Driven Models: The paper provides a complementary Bayesian interpretation to existing information-theoretic justifications. It posits that score-driven models act as "plug-in" analogues to Tweedie-style Bayesian updating. In this view, the intractable marginal observation score (required for exact Bayesian correction) is replaced by the tractable conditional likelihood score evaluated at the predictive parameter.
Local Bayesian-Fisher Approximation (Proposition 1): The authors prove that for general conditional densities, the inverse-Fisher-scaled conditional score arises naturally as a local Bayesian correction. This result unifies the concepts of Fisher scoring, Gaussian predictive approximations, and precision discounting, offering a local Bayesian justification for why score-driven updates take their specific form.
Exact Equivalence in Expectation Space (Proposition 2): A central result is the identification of conditions under which the score-driven update is not merely an approximation but an exact Bayesian update. Specifically, for natural exponential families with conjugate priors and steady-state precision discounting, the score-driven correction in the expectation space coincides algebraically with the Bayesian posterior mean. This equivalence holds when the scaling matrix is the inverse Fisher information and the update is applied to the expectation parameter.
Role of Variance Functions and Tweedie's Index: The paper clarifies the role of Tweedie's variance-function index in normalizing forecast errors. It distinguishes between the normalization inherent in the likelihood score (via the variance function) and the additional scaling applied in the recursion. The analysis shows how different scaling exponents (e.g., $d=0, 1/2, 1$ ) affect the preservation of information weighting, with $d=1$ (inverse-Fisher scaling) aligning with the exact conjugate Bayesian update in expectation space.
Distinction Between Scores: The paper rigorously distinguishes between the marginal score of the predictive density (central to Tweedie's formula) and the conditional score of the likelihood (central to score-driven models). It explains that while the former is the exact Bayesian correction, the latter serves as a computationally feasible substitute that becomes exact only under specific conjugacy and parameterization conditions.

Significance and Claims
The paper claims to link empirical Bayes denoising, approximate filtering, conjugate Bayesian updating, and observation-driven time-series modeling. Its significance lies in providing a "Bayesian lens" that clarifies the relationship between score-driven models and exact inference:

Clarification of Approximation vs. Exactness: The results distinguish when a score-driven update is a local approximation (general densities) versus when it is an exact algebraic identity (conjugate NEF in expectation space).
Parameterization Sensitivity: The paper highlights that the exact equivalence is sensitive to parameterization. While the equivalence holds in the expectation space for conjugate families, it generally fails if the recursion is written in the natural parameter or a transformed parameter (e.g., log-mean), even if the underlying model is conjugate.
Complementarity: The authors emphasize that this Bayesian interpretation does not replace existing foundations (such as Kullback-Leibler optimality or robust filtering) but complements them. It identifies specific settings where the score correction has a precise signal-extraction interpretation, thereby explaining why inverse-Fisher scaling is a natural choice in many empirical applications while acknowledging that alternative scalings may be preferable for stability or robustness in non-conjugate or heavy-tailed contexts.

In summary, the paper establishes that score-driven models can be viewed as tractable, recursive implementations of Tweedie's formula, where the intractable marginal score is substituted by a conditional score, and the exactness of this substitution is guaranteed only under specific conjugate and steady-state conditions in expectation space.