📊 statistics

Value of Information under Imprecise Probabilities

This paper proposes a generalized Value of Information (IP-VOI) framework that extends conventional analysis to settings with imprecise probabilities by evaluating decision stability across an admissible set of probability measures, thereby distinguishing robust research conclusions from those dependent on specific modeling assumptions.

Original authors: Rowan Iskandar

Published 2026-07-20

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Rowan Iskandar

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Art of Guessing the Future

Imagine you are the captain of a ship trying to decide whether to sail around a dangerous reef or risk a shortcut through a foggy channel. You have a map, but it's a bit blurry. Maybe the reef is actually a sandbar, or maybe the fog is thicker than you think. In the world of science and medicine, this is exactly what decision-makers face every day. They use "decision models"—mathematical maps—to choose the best treatment for patients or the best policy for a country. But these maps rely on guesses about the future, like how likely a drug is to work or how much it will cost.

To make these guesses smarter, scientists use a tool called Value of Information (VOI). Think of VOI as a "price tag" for knowledge. It asks: "If we could know the answer to a specific question right now (like the exact success rate of a drug), how much money or health would we save by making a better choice?" If the answer is "a lot," it makes sense to spend money on a new study to find that answer. If the answer is "not much," we should stop guessing and just pick the best option we have.

Traditionally, to calculate this price tag, scientists have to pick one specific map of the future. They have to say, "We are 100% sure the fog is this thick." But in the real world, evidence is often messy. Sometimes, the data supports several different maps at once, and no single one is clearly the winner. This is where the new paper by Rowan Iskandar steps in, offering a way to handle the fog without pretending it's gone.

The Problem with Picking Just One Map

The paper tackles a tricky problem: What happens when the evidence doesn't point to just one "perfect" map, but rather a whole family of defensible maps?

Imagine you are trying to guess the weight of a mystery box.

The Old Way: You pick one guess (say, 10 pounds), calculate the cost of being wrong, and decide if you should buy a scale to weigh it.
The Reality: The evidence actually supports a range of possibilities. The box could be 8 pounds, 10 pounds, or 12 pounds, and you can't prove which one is right. If you just pick 10 pounds, you might think the scale is worth buying. But if the box is actually 8 pounds, maybe the scale isn't worth it at all.

The author argues that forcing a single guess when you have a range of possibilities is like wearing blinders. It might make the math look clean, but it hides the fact that your conclusion depends entirely on which guess you happened to pick.

The New Approach: The "Foggy Envelope"

Iskandar introduces a new method called Imprecise-Probability Value of Information (IP-VOI). Instead of picking one map, this method keeps all the defensible maps on the table at once. It creates a "foggy envelope" that wraps around all the possible answers.

Here is how it works in the paper's own "wound-dressing" example (a study about how to dress surgical wounds to prevent infection):

The Setup: The researchers looked at a decision about which type of dressing to use. They identified 36 different ways to interpret the evidence. Some maps assumed the infection risk was high; others assumed it was low. Some assumed the treatments were linked in a certain way; others didn't.
The Old Result: If you pick just one "standard" map, the math says the "Expected Value of Perfect Information" (how much we'd gain from knowing everything) is about £152 per wound. This suggests a massive national study is worth it.
The New Result: When the researchers looked at the envelope of all 36 maps, the value of that information wasn't a single number. It was a range: £74 to £249 per wound.

This range tells a much richer story.

If the decision threshold (the amount of money you need to save to justify a study) is £50, the study is worth it no matter which map is true. The whole envelope is above the line.
If the threshold is £100, the envelope straddles the line. This means the answer is assumption-dependent. If the infection risk is low, the study isn't worth it. If it's high, it is. A single-map analysis would have missed this uncertainty and might have led to a wrong decision.

The "Guaranteed" vs. The "Best Guess"

The paper also introduces a clever way to make a decision even when you are unsure. It uses a rule called the "lower-expectation" (or "worst-case" thinking). Instead of asking, "What is the average gain?" it asks, "What is the guaranteed gain, even if the worst possible map turns out to be true?"

In the wound-dressing example:

The "average" gain from a perfect study was £152.
The "guaranteed" gain (the value you get even if the evidence is on the pessimistic side) was £122.

This is a crucial distinction. It helps decision-makers see if a recommendation is robust (stable across all possibilities) or fragile (only works if you pick the most optimistic map).

What the Paper Finds (and What It Doesn't)

The main finding is that this new method doesn't just give a different number; it changes the kind of question we ask.

It finds: In the national-scale wound-dressing case, the conclusion that "a study is worth it" was robust. Even the worst-case map said the study was valuable.
It finds: However, for smaller groups (like a single hospital), the conclusion became assumption-dependent. The envelope showed a "break-even" zone where the decision flipped depending on which map you believed.
It rules out: The paper argues against the idea that you can always just pick one "best" map and ignore the rest. It shows that doing so can give a misleading impression of certainty. If your single-map result is £150, but the envelope is £74–£249, you don't know if you are safe or in trouble.
It clarifies: This isn't a magic bullet that solves the uncertainty. The paper admits that calculating this is computationally expensive (it takes a lot of computer power to run 36 simulations instead of one) and that the results are simulations, not absolute proofs. It also clarifies that this method is for when you cannot assign probabilities to the different maps (i.e., you don't know how likely each map is). If you can assign those weights, the old methods still work.

The Takeaway

This paper is like handing a decision-maker a new pair of glasses. The old glasses let them see a single, sharp line where the answer is. The new glasses show a fuzzy, shaded zone. While a fuzzy zone might look less satisfying at first, it is actually more honest. It tells the decision-maker: "Here is the range of what the evidence supports. If your decision is safe anywhere in this zone, go ahead. If your decision only works on the very edge of the zone, be careful—you might be betting on a guess, not a fact."

By using this "envelope" approach, scientists can stop pretending they know more than they do and start making research decisions that are truly robust, no matter how the fog clears.

Technical Summary: Value of Information under Imprecise Probabilities

1. Problem Statement

Conventional Value of Information (VOI) analysis, including Expected Value of Perfect Information (EVPI), Expected Value of Partial Perfect Information (EVPPI), and Expected Value of Sample Information (EVSI), relies on the specification of a single, precise probability measure for uncertain model inputs. However, in applied decision modeling—particularly in health technology assessment—evidence often fails to uniquely determine a single distribution. Instead, evidence may support multiple defensible distributional forms, dependence assumptions, extrapolation models, or structural choices.

When evidence supports a set of plausible measures but does not justify assigning specific weights (probabilities) to them, selecting a single "reference" measure introduces an unsupported precision. This can lead to misleading conclusions regarding research prioritization, where a decision to fund or reject a study appears robust based on one measure but is actually highly sensitive to the specific modeling assumptions chosen. The paper addresses the need to extend VOI calculations to settings characterized by imprecise probabilities, where uncertainty is represented by an admissible set of probability measures rather than a single point estimate.

2. Methodology

The paper proposes a framework for Imprecise-Probability Value of Information (IP-VOI) that separates the analysis into three distinct components: the admissible set of probability measures, the decision rule used to choose among strategies under imprecision, and the information target.

2.1 The Admissible Set and P-Boxes

Instead of a single measure $P_0$ , the analyst defines a non-empty set of admissible probability measures $\mathcal{P}$ compatible with current evidence and modeling assumptions. This set may be finite (e.g., a Cartesian product of alternative structural models) or continuous (e.g., parameters within a range).

Implementation: The paper utilizes Probability Bounds Analysis (PBA) and p-boxes (pairs of bounding cumulative distribution functions) as a practical route to encode and propagate these measures through decision models.
Distinction: The framework distinguishes between variability (aleatory uncertainty, described by distributions) and imprecision (epistemic uncertainty, described by bounds on those distributions).

2.2 Two Complementary Quantities

The framework defines two distinct outputs to address different decision-making needs:

Rule-Specific IP-VOI:
- This calculates the value of information under a specific decision rule applied to the set $\mathcal{P}$ .
- The paper primarily implements the lower-expectation (Γ-maximin) rule, which selects the strategy with the highest guaranteed expected net benefit across all measures in $\mathcal{P}$ .
- Formally, for a target $Z$ (perfect, partial, or sample information), the value is:
  $VOI^\rho_Z = V^\rho_Z(\mathcal{P}_Z) - V^\rho_0(\mathcal{P})$
  where $V^\rho$ is the value under rule $\rho$ . For EVPI, this simplifies because the pointwise optimal strategy is the same for all measures in $\mathcal{P}$ . For EVPPI and EVSI, a finite policy search is required as the optimal policy may vary across measures.
Fixed-Measure VOI Envelope:
- This is a diagnostic tool, not a decision rule. It reports the range $[ \underline{T}, \overline{T} ]$ of conventional VOI values obtained by evaluating the standard VOI calculation for every measure $P \in \mathcal{P}$ .
- Classification: The envelope allows researchers to classify research decisions relative to a threshold $\tau$ $τ$ :
  - Robust: The entire envelope lies on one side of $\tau$ (e.g., all measures support the research).
  - Assumption-dependent: The envelope straddles $\tau$ (some measures support the research, others do not).
- This identifies when a single-measure conclusion is misleading because it lies on one side of the threshold while the evidence-supported set crosses it.

2.3 Unified Algorithm

The paper provides a unified computational algorithm that:

Discretizes the admissible set $\mathcal{P}$ into an index set $H$ .
Runs an inner Monte Carlo simulation for each measure in $H$ .
Aggregates results via outer minimization/maximization (for rule-specific values) or range extraction (for envelopes).
Includes a low-variance nested estimator for EVSI to reduce computational noise.

3. Key Results (Wound-Dressing Case Study)

The framework was demonstrated using a transparent reconstruction of the BLUEBELLE wound-dressing decision model, comparing four strategies for surgical-site infection prevention. The admissible set $\mathcal{P}$ consisted of 36 measures derived from varying four dimensions: baseline infection-risk sources, health loss values, treatment-effect evidence specifications, and correlations among estimates.

Stability of Strategy: Across all 36 measures, the "complex dressing" strategy remained the preferred option, and the lower-expectation IP-EVPI was approximately £122 per wound.
Envelope Width: The fixed-measure EVPI envelope ranged from £74 to £249 per wound. The reference measure (£152) fell within this range but was not the midpoint.
Threshold Sensitivity:
- At thresholds of £50 or £260, the conclusion (whether research is worthwhile) was stable across the entire set.
- At thresholds of £100 or £200, the conclusion was assumption-dependent. A single-measure analysis would have provided a definitive "yes" or "no," whereas the IP-VOI envelope revealed that the decision depends on which admissible measure is selected.
Research Commissioning: For a national-scale trial, the lower bound of the EVSI envelope (£0.47 billion) exceeded plausible costs, certifying the recommendation as robust. However, the envelope identified specific "break-even" population sizes (e.g., 7,600–38,300 wounds for a £1.75M trial) where the decision becomes assumption-dependent.

4. Significance and Claims

The paper claims that IP-VOI generalizes standard VOI to settings where evidence does not identify a single precise probability measure. Its primary significance lies in:

Transparency: It forces the analyst to explicitly state the set of defensible measures and the decision rule used, rather than hiding imprecision behind a single arbitrary distribution.
Diagnostic Power: The fixed-measure envelope serves as a critical diagnostic for research prioritization. It distinguishes between conclusions that are stable across the evidence-supported set and those that rely on unresolved modeling assumptions.
Avoiding Misleading Certainty: It prevents the "misleading impression of certainty" that arises when a single-measure VOI value is reported in isolation, particularly when the decision threshold falls within the range of plausible values.
Implementation: The paper provides a unified algorithm and executable code (R/Quarto and Python) to implement these calculations, making the approach accessible for applied decision modeling.

The authors emphasize that IP-VOI is not a replacement for model averaging (which requires defensible weights) but addresses the complementary case where such weights cannot be justified. It does not claim to solve all structural uncertainty issues but provides a rigorous method to quantify how much research value varies when the probability measure itself is uncertain.