📊 statistics

Revisiting the Lost Submarine Problem: A Decision Theoretic Approach

This paper argues that the criticisms of the "lost submarine problem" raised by Morey et al. (2016) regarding the limitations of confidence intervals can be resolved through a decision-theoretic approach, which demonstrates that defining a procedure's specific purpose yields a single optimal choice, thereby framing the existence of diverse statistical methods as an advantage rather than a flaw.

Original authors: Anthony Almudevar

Published 2026-02-02

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Anthony Almudevar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A Rescue Mission Gone Wrong (Statistically)

Imagine a submarine is lost underwater. It's sitting still, and we know it's exactly 2 units long. We don't know where it is, but we know its "hatch" (the rescue point) is exactly in the middle.

Suddenly, two bubbles pop up from the water. We know these bubbles came from random spots along the length of the submarine. By measuring where these two bubbles are, we can draw a line and narrow down where the submarine might be.

The Problem:
We need to tell the rescue team exactly where to dive. We have a best guess (the middle of the two bubbles), but we also need to tell them: "How sure are we?"

In statistics, the standard way to answer "How sure are we?" is to give a Confidence Interval. This is like drawing a box around your guess and saying, "We are 95% confident the submarine is inside this box."

The Conflict:
The paper points out a famous puzzle: If you ask four different statisticians to draw this "box" using standard rules, they will all give you four different boxes.

Statistician A draws a wide box.
Statistician B draws a narrow box.
Statistician C draws a box that changes shape based on how far apart the bubbles are.

The paper argues that this isn't a bug; it's a feature. The reason they are different is that the statisticians are answering slightly different questions. The paper uses a "Decision Theoretic" approach (thinking about the cost of being wrong) to figure out which box is actually the best one to use.

The Four "Boxes" (Confidence Intervals)

The author reviews four common ways to draw this box. Think of them as four different strategies for packing a suitcase:

The "Average" Strategy (SD): This ignores the specific distance between the bubbles and just uses a standard rule based on how the math usually works. It's like packing a suitcase the same size every time, regardless of what you're actually carrying.
The "Nonparametric" Strategy (NP): This looks at the bubbles and says, "If the bubbles are far apart, the submarine must be huge, so we need a bigger box." It adjusts the box size based on the data.
The "Super-Test" Strategy (UMP): This uses a very strict mathematical test to find the smallest possible box that still guarantees safety. It's like a master packer who squeezes everything in perfectly.
The "Bayesian" Strategy (BC): This assumes a "prior" belief (like a hunch) about where the submarine might be and calculates the box based on that.

The Flaw:
The paper shows that two of these strategies (SD and NP) are "inadmissible." In plain English, this means they are wasteful. They sometimes draw a box that is wider than necessary, even though the math proves the submarine cannot be outside a smaller area. It's like drawing a map that says "The treasure is somewhere in this entire ocean," when you actually know it's only in this specific bay.

The Solution: Ask "What is the Goal?"

The author argues that there is no single "best" box unless you define what you are trying to achieve. This is the "Decision Theoretic" part.

Imagine you are the rescue captain. You have to decide how much time and fuel to spend searching. The "cost" of your search depends on the size of the box you are given.

The paper explores two different goals (Loss Functions):

Goal 1: "Save Time and Fuel" (Minimum Search Effort)

The Scenario: The rescue team has limited fuel. They want to search the smallest possible area that still has a 50% chance of finding the sub.
The Result: The paper calculates that the best strategy is surprising. You should only search when the bubbles are very far apart (which means the sub is definitely in a small area). If the bubbles are close together, you should just pick a tiny spot and hope for the best, because the math says the "safe" area is huge and searching it would waste too much fuel.

Analogy: Imagine looking for a lost dog. If you see two paw prints far apart, you know the dog is in a small park. You search the park. If the paw prints are right next to each other, the dog could be anywhere in the whole city. Searching the whole city is too expensive, so you just pick one spot and hope.

Goal 2: "Don't Waste Time if We're Wrong" (Minimum Effort Conditional on Being Right)

The Scenario: The captain says, "I don't care how big the box is if we are wrong. But if we are right, I want the box to be as small as possible so we don't waste time."
The Result: This leads to a completely different strategy. The best box is the one that stays the same size regardless of the bubbles (until it hits the physical limit of the submarine).

Analogy: This is like betting on a horse race. If you lose, you don't care how much you bet. But if you win, you want to have bet the minimum amount necessary to win.

The Main Takeaway

The paper concludes that the confusion in statistics (why there are so many different confidence intervals) isn't a failure of math. It's because different problems require different solutions.

If you want to minimize the average size of your search area, use Strategy A.
If you want to minimize the size of your search area only when you are correct, use Strategy B.

The author's final message is: Stop asking "Which confidence interval is the right one?" and start asking "What is the cost of being wrong in my specific situation?" Once you define the cost (the "loss function"), the math will tell you exactly which interval to use. There is no single "magic" interval; there is only the best interval for your specific goal.

Technical Summary: Revisiting the Lost Submarine Problem: A Decision Theoretic Approach

Problem Statement
The paper revisits the "Lost Submarine Problem," originally highlighted by Morey et al. (2016) to illustrate the limitations of relying on confidence intervals (CIs) as formal inference statements. The scenario involves a stationary submarine of length $2K$ with a hatch at its center, $\theta$ . Two bubbles emerge from random locations along the submarine's length, providing independent observations $X_1, X_2$ uniformly distributed on $[\theta - K, \theta + K]$ . The objective is to estimate the hatch position $\theta$ and assess the precision of this estimate.

While the point estimator $\hat{\theta} = (X_1 + X_2)/2$ is widely accepted as the optimal location-equivariant estimator minimizing risk under symmetric loss functions, the construction of a confidence interval for $\theta$ with a fixed coverage level (e.g., $1-\alpha = 0.5$ ) yields disparate results depending on the underlying statistical principle used. The paper investigates why these procedures differ and proposes a framework to resolve this ambiguity.

Methodology
The author employs a decision-theoretic framework to analyze the construction of confidence intervals. The methodology proceeds in three stages:

Distributional Analysis: The problem is transformed using the sample mean $M = (X_1 + X_2)/2$ and the range $V = |X_2 - X_1|$ . $M$ serves as the estimator for $\theta$ , while $V$ is an ancillary statistic. The conditional distribution of $M$ given $V=v$ is uniform on $[\theta - (1 - |v|/2), \theta + (1 - |v|/2)]$ (assuming $K=1$ ). This establishes that the precision of the estimate depends entirely on the observed value of the ancillary statistic $V$ .
Review of Conventional Procedures: Four distinct methods for constructing a $1-\alpha$ confidence interval are reviewed:
- Sampling Distribution (SD): Based on the marginal distribution of $\hat{\theta} - \theta$ .
- Nonparametric (NP): Based on the event that $\theta$ lies between the two observations.
- Uniformly Most Powerful (UMP): Derived from the Neyman-Pearson lemma for testing hypotheses about $\theta$ .
- Bayesian Credible (BC): Based on the central portion of the likelihood support.
  The paper demonstrates that these methods yield intervals of the form $\hat{\theta} \pm b(V)$ with significantly different functional forms for the half-width $b(V)$ .
Admissibility and Optimization:
- Admissibility: The paper establishes that any confidence interval with a half-width $b(V)$ exceeding the theoretical maximum possible error, $(1 - |V|/2)$ , is inadmissible. Such intervals can be truncated to the admissible bound without losing coverage probability.
- Decision Theoretic Formulation: Recognizing that multiple admissible procedures remain, the author introduces a loss function $L_\theta(L, U)$ to define the "purpose" of the interval. The risk is minimized by selecting the function $b(V)$ that minimizes the expected loss subject to the coverage constraint.

Key Results
The application of the decision-theoretic approach yields two distinct optimal procedures based on different definitions of "loss" or "cost":

Minimum Search Effort: If the goal is to minimize the expected length of the search interval (assuming the search effort is proportional to the interval length), the optimal strategy is to allocate the confidence level entirely to the scenarios where the interval is shortest. The resulting optimal half-width function is:
$b(v) = \begin{cases} (1 - |v|/2) & \text{if } |v| \geq k_\alpha \\ 0 & \text{if } |v| < k_\alpha \end{cases}$
where $k_\alpha = 2(1 - \sqrt{1-\alpha})$ . This procedure is discontinuous and differs radically from conventional methods.
Minimum Search Effort Conditional on Correctness: If the objective is to minimize the expected length only when the interval actually contains the true parameter (i.e., minimizing the cost of a successful rescue), the loss function changes. The optimal half-width function becomes:
$b(v) = \begin{cases} (1 - |v|/2) & \text{if } |v| \geq \sqrt{4\alpha} \\ k_\alpha & \text{if } |v| < \sqrt{4\alpha} \end{cases}$
where $k_\alpha = 1 - \sqrt{\alpha}$ . This solution corresponds to the Sampling Distribution (SD) procedure (after applying the admissibility bound).

Significance and Claims
The paper argues that the existence of multiple confidence intervals is not a flaw in statistical theory but a reflection of the fact that different statistical procedures serve different purposes. The "futility" of relying on a single confidence interval as a universal inference statement arises only when the specific decision context is ignored.

The author claims that once the purpose of the inference is precisely defined via a loss function, a single optimal choice emerges. Consequently, the variety of existing procedures is an advantage, allowing practitioners to select the method that aligns with their specific decision-theoretic goals (e.g., minimizing search time vs. minimizing error conditional on success). The paper concludes that the careful selection of the loss function is the most critical aspect of the inference problem, resolving the apparent contradictions in the Lost Submarine Problem without discarding the concept of confidence intervals.