📈 economics

How to Compare Copula Forecasts?

This paper proposes a principled framework for comparing copula forecasts by introducing novel multi-objective scores that overcome the inherent non-elicitability of copulas, allowing for a two-step testing procedure that distinguishes between the predictive accuracy of the copula and its marginal distributions.

Original authors: Tobias Fissler, Yannick Hoga

Published 2026-02-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tobias Fissler, Yannick Hoga

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge at a high-stakes cooking competition. You have two contestants, and you want to decide who is the better chef.

However, there is a catch: one chef is amazing at making the main course (the meat), but mediocre at the sauce (the seasoning). The other chef is a master of the sauce, but their main course is just okay.

If you just give them one overall score from 1 to 10, you might end up picking the wrong winner. You won't know if the high score was because of the meat or the sauce. To be a fair judge, you need a way to score them that lets you say, "Chef A wins because their meat was better, even though their sauce was slightly worse than Chef B's."

This paper is about a mathematical version of that problem.

The Problem: The "Copula" Dilemma

In statistics, when we try to predict how multiple things move together (like how five different stock markets crash at the same time), we use something called a Copula.

A Copula is like the "sauce" of a statistical model. It describes the relationship or the connection between variables. But a complete prediction also needs the "main course"—the individual behavior of each variable (called the Marginals).

For a long time, mathematicians have struggled with a problem: You can't judge the "sauce" (the Copula) by itself.

If you try to score a Copula without looking at the individual variables, the math breaks. It’s like trying to judge a sauce without knowing if it's being poured over a steak or a piece of fish. Because the "sauce" depends on the "meat," you can't give the sauce a fair, consistent grade on its own.

The Discovery: The "Non-Elicitability" Trap

The authors first prove something negative: Copulas are "non-elicitable."

In plain English, this means there is no single, perfect "ruler" that can measure a Copula's accuracy in isolation. If you try to create one, the math becomes inconsistent. You could end up rewarding a "bad" forecast just because the "meat" happened to be perfect that day.

The Solution: The Two-Step "Multi-Objective" Judge

The authors propose a brilliant workaround. Instead of trying to find one ruler, they create a two-step scoring system using something called "Multi-Objective Elicitability."

Think of this as a judge with two different clipboards:

Clipboard 1 (The Marginals): First, the judge scores how good the "main course" was for each chef.
Clipboard 2 (The Copula): Only if the "main courses" are equally good does the judge look at the second clipboard to score the "sauce."

By using a special mathematical ranking (called Lexicographic Order), the judge can prioritize the "meat" first. If one chef's meat is significantly better, the judge declares them the winner immediately. But if the meat is a tie, the judge uses the sauce score to break the tie.

Why does this matter?

This isn't just math for math's sake; it has real-world consequences for money and risk.

If a bank is trying to predict if several global markets will crash at once, they use these models. If their model is wrong, they could lose billions.

Before this paper, if a bank's model failed, the experts wouldn't know why. Was the model bad at predicting individual stocks (the meat)? Or was it bad at predicting how those stocks move together (the sauce)?

This paper gives them a "diagnostic tool." It allows them to say: "Our model's individual predictions are great, but our connection logic is flawed. We need to fix the sauce."

Summary in a Nutshell

The Old Way: Trying to grade the "connection" between variables without accounting for the variables themselves (which leads to unfair and inconsistent results).
The New Way: A two-step "tie-breaker" system that grades the individual parts first, and then uses the "connection" to decide the winner if the parts are equal. This allows scientists to pinpoint exactly where their predictions are failing.

1. The Problem: The Non-Elicitability of Copulas

In forecasting, the gold standard for model comparison is the use of strictly consistent scoring functions. A functional (like the mean or a copula) is "elicitable" if there exists a scoring function that is uniquely minimized in expectation by the true value.

The authors identify a critical theoretical obstacle: Copulas are generally not elicitable.

Technical Reason: Elicitability requires the functional to have Convex Level Sets (CxLS). The authors prove (Proposition 4) that the copula functional fails this property.
Practical Implication: Because they are not elicitable, copula forecasts cannot be sensibly compared on their own. If one copula model produces a lower score than another, there is no theoretical guarantee that the lower score implies a more accurate dependence structure, especially if the marginal distributions are misspecified.

2. Methodology: Multi-Objective Elicitability

To solve this, the authors move away from scalar scoring and adopt a multi-objective approach. They draw inspiration from the strategy of pairing non-elicitable functionals with auxiliary ones (e.g., pairing Variance with Mean).

A. Fréchet Classes (The Special Case)

The authors note that if the marginal distributions are fixed and known (a Fréchet class), the copula becomes elicitable. In this restricted scenario, standard scoring rules (like the Kullback–Leibler Information Criterion) are formally justified.

B. Multi-Objective Scoring (The General Case)

When marginals are not fixed, the authors propose pairing the copula with the marginal distributions. To avoid the "attribution problem" (not knowing if a bad score is due to the copula or the marginals), they introduce strictly consistent multi-objective scores using lexicographic ordering ( $\preceq_{lex}$ ).

The proposed bivariate score $S$ for a copula $C$ and marginals $\{F_i\}$ is:
$S(C, \{F_i\}, y) = \begin{pmatrix} S_{marg}(\{F_i\}, y) \\ S_{cop}(C, \{F_i\}, y) \end{pmatrix}$
Under lexicographic order, the first component (marginals) is prioritized. A forecast is only judged on its copula component if the marginal components are equally accurate.

3. Key Contributions

Negative Result: Formal proof that copulas lack the CxLS property and are thus generally non-elicitable.
Positive Result (Theorem 10): Proof that the tuple of (copula, marginals) is multi-objective elicitable. This provides a "truth serum" for joint forecasting.
Two-Step Hypothesis Testing: The authors develop a testing procedure to compare two models ( $F_1$ $F_{1}$ vs $F_2$ $F_{2}$ ).
- Step 1: Test if marginals have equal predictive accuracy.
- Step 2: If marginals are equal, test if copula forecasts have equal or superior predictive ability.
- Critical Values: They derive specific critical values ( $c_{1n}, c_{2n}$ ) to ensure the overall asymptotic size $\alpha$ is controlled across both steps.

4. Results and Empirical Evidence

Simulation Results

The authors used a CCC-GARCH data-generating process to test their methodology.

Size and Power: The two-step tests correctly controlled the size (rejection rate near $\alpha=5\%$ under the null) and demonstrated high power in detecting differences in both marginals and copulas.
Attribution: The tests successfully "attributed" rejections to the correct component. If only marginals differed, the first step rejected; if only copulas differed, the second step rejected.

Empirical Application

The methodology was applied to international stock market indices (S&P 500, DAX, etc.) comparing GARCH-type models (DCC, CCC) against GAS models.

Findings: The tests revealed that while several models produced similar marginal density forecasts, the DCC-GARCH models significantly outperformed GAS models in capturing the dependence structure (copula component).
Utility: The authors demonstrated that their method could distinguish between models that a standard log-score (which sums marginal and copula errors) would conflate.

5. Significance

This paper is significant because it provides the first mathematically rigorous way to perform performance attribution in multivariate forecasting.

By separating the accuracy of the "shape" (copula) from the "scale/location" (marginals), practitioners in finance (e.g., Value-at-Risk modeling) and meteorology can determine whether a model's failure is due to a poor understanding of individual variables or a poor understanding of how those variables interact.