🔢 mathematics

Shannon meets Gödel-Tarski-Löb: Undecidability of Shannon Feedback Capacity for Finite-State Channels

This paper proves that determining whether the feedback capacity of a finite-state channel meets a specific rational threshold is an undecidable problem, establishing a fundamental limit on exact capacity reasoning and linking the issue to Gödel-Tarski-Löb incompleteness phenomena.

Original authors: Angshul Majumdar

Published 2026-03-19

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Angshul Majumdar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build the ultimate "perfect radio" for a very specific, tricky type of communication channel. This channel has a memory (it remembers what happened before) and a "state" (like a hidden mode it switches between). You also have a "feedback" loop, meaning the receiver can tell the sender what it heard, allowing the sender to adjust its strategy in real-time.

The goal of this paper is to answer a very specific question: "Can we write a computer program that tells us, with 100% mathematical certainty, if this perfect radio can achieve a data speed of at least 50%?"

The author, Angshul Majumdar, says: No. It is impossible.

Here is the breakdown of why, using simple analogies.

1. The "Perfect Radio" vs. The "Infinite Horizon"

In the world of communication, we often look at how much data we can send over a short time (like a 10-second clip). We can calculate this easily. But "Capacity" is about the long run. It's asking: "If we keep sending data forever, what is the absolute maximum speed we can sustain?"

The paper asks: If I give you the blueprints for a specific channel (with numbers that are simple fractions), can you write a program that looks at those blueprints and says, "Yes, the long-term speed is definitely above 50%" or "No, it's definitely below 50%"?

2. The "Trick" of the Delayed Switch

To prove this is impossible, the author builds a "trap" using two very similar channels. Imagine two identical-looking cars:

Car A (The Good One): It drives silently for 1,000 miles, then suddenly turns into a super-fast sports car that can go 100 mph forever.
Car B (The Bad One): It drives silently for 1,000 miles, then turns into a broken-down truck that can only go 0 mph forever.

The Problem: If you only look at the first 1,000 miles (the "finite horizon"), both cars look exactly the same. They are both silent and slow. You cannot tell them apart.

However, the "Capacity" of the channel is defined by what happens forever.

Car A has a high capacity (100 mph).
Car B has zero capacity.

Because the "good" and "bad" versions can look identical for any amount of time you choose to test them, a computer program can never be sure which one it is looking at just by running a simulation. It might run for a million years, see them both acting the same, and still not know if the "switch" to the fast/slow mode is coming tomorrow or never.

3. The "Gödel-Tarski-Löb" Connection (The Logic Trap)

The title mentions three famous logicians: Gödel, Tarski, and Löb. You can think of them as the "Rule Makers" of mathematics.

Gödel proved that in any complex enough system of rules, there are true statements that you can never prove using those rules.
Tarski proved you can't define "truth" inside the system itself.
Löb showed limits on how a system can trust its own proofs.

The paper shows that the question "Is the capacity above 50%?" is one of those unprovable statements.
If you try to build a "Universal Theory of Communication" that claims to solve this for every possible channel, that theory will inevitably fail. It will either:

Give the wrong answer sometimes.
Get stuck forever trying to calculate the answer.
Be unable to prove the answer even if it knows it's true.

4. What This Does NOT Mean

It is important not to be too pessimistic. This paper does not say:

"We can't calculate capacity for any channel." (We can for many simple ones).
"We can't get close to the answer." (We can use approximations).
"We can't send data." (We can still communicate).

It simply says: There is no single, universal "magic button" algorithm that can solve this exact problem for every possible variation of these channels.

The Big Takeaway

Think of this like a map.

Old View: We thought, "If we just make the map detailed enough, we can navigate any terrain perfectly."
New View (This Paper): "Actually, for some terrains, no matter how detailed the map is, you can never know for sure if the path leads to a dead end or a treasure chest without walking the whole infinite path. And since you can't walk an infinite path in finite time, you can never write a rule that guarantees the answer."

In short: The author proves that for a broad class of communication channels, the question of "What is the exact maximum speed?" is a mathematical mystery that no computer can ever solve with 100% certainty. We must rely on approximations or specific, simpler cases instead of a universal solution.

1. Problem Statement

The paper addresses the exact decision problem for the feedback capacity of Finite-State Channels (FSCs). Specifically, it investigates whether a universal algorithm exists to determine if the feedback capacity of a given channel exceeds a specific rational threshold.

Input:
- An effective encoding $e$ of a binary-input, binary-output, rational, unifilar finite-state channel ( $W_e$ ) with a specified rational initial state distribution ( $\pi_{1,e}$ ).
- A rational threshold $q \in \mathbb{Q}$ .
Question: Does the exact infinite-horizon feedback capacity satisfy $C_{fb}(W_e, \pi_{1,e}) \geq q$ ?
Goal: Determine the decidability of the predicate $Cap(e, q)$ and the associated language $L_{Cap}$ .

The paper explicitly distinguishes this from approximation problems, finite-horizon surrogates, or promise-gap variants. It focuses strictly on the exact, infinite-horizon threshold.

2. Methodology

The author employs a multi-layered approach combining Information Theory, Computability Theory, and Mathematical Logic.

A. Structural Construction (The "Delayed-Activation" Family)

To prove undecidability, the paper constructs a specific family of channels within the restricted class of rational unifilar FSCs.

Mechanism: A "delayed-activation" mechanism is used where the channel state increments deterministically for $N$ steps (the "delay phase") before entering an absorbing "active" state ( $\star$ ).
Two Variants:
1. $Ch(N)_{good}$ : Once in the active state, the channel becomes noiseless ( $Y = X$ ).
2. $Ch(N)_{bad}$ : Once in the active state, the channel becomes completely noisy (output is independent of input).
Key Property: For any finite horizon $n \leq N+1$ , both channels produce identical output sequences (deterministically 0) and thus have identical finite-horizon directed information values ( $V_n = 0$ ). However, their asymptotic feedback capacities differ drastically ( $C_{fb} = 1$ for good, $C_{fb} = 0$ for bad).

B. Reduction from Undecidable Problems

The proof utilizes a many-one reduction from a known undecidable problem (e.g., the Halting Problem or a specific tiling problem) to the capacity threshold problem.

A computable function maps an instance of the undecidable problem to a channel encoding $e$ and a threshold $q$ .
The construction ensures that the "YES" instance of the source problem corresponds to a channel with high capacity, and the "NO" instance corresponds to a channel with zero capacity.
Because the source problem is undecidable, the capacity threshold problem must also be undecidable.

C. Logical and Algebraic Analysis

$\exists\mathbb{R}$ Barrier: The paper investigates whether the problem lies in the complexity class $\exists\mathbb{R}$ (Existential Theory of the Reals), which covers problems reducible to systems of polynomial inequalities. Since $\exists\mathbb{R}$ is decidable (in PSPACE), proving the problem is undecidable implies it cannot be reduced to $\exists\mathbb{R}$ .
Gödel-Tarski-Löb (GTL) Analysis: The paper formalizes the capacity threshold as an arithmetic predicate within a recursively axiomatizable theory ( $T$ ). It then applies classical meta-mathematical theorems to derive consequences regarding provability and truth.

3. Key Contributions and Results

1. Undecidability of Exact Thresholding

Theorem IV.4: The exact feedback-capacity threshold problem is undecidable even when restricted to the severely constrained class of binary-input, binary-output, rational, unifilar FSCs.

Implication: There is no algorithm that can universally decide if $C_{fb} \geq q$ for all such channels. This holds despite the channels having finite state spaces and rational parameters.

2. Barrier to Universal Finite-Letter Representations

Corollary IV.7: There is no uniform finite-horizon characterization of feedback capacity.

Even if one computes the directed information $V_n$ for any fixed $n$ , it is insufficient to determine the asymptotic capacity $C_{fb}$ . The behavior of the channel can change arbitrarily far in the future (after $N$ steps) without affecting any finite-horizon metric up to $N$ .
Consequently, $C_{fb}$ cannot be represented as a finite-letter entropic functional (a single-letter formula or a fixed-depth dynamic program) valid for the entire class.

3. Non-Membership in $\exists\mathbb{R}$

Theorem V.7: The language $L_{Cap}$ is not in $\exists\mathbb{R}$ .

This means the exact capacity threshold cannot be captured by a universal system of existential polynomial equalities and inequalities over the reals. This places the problem strictly outside the standard framework used for many geometric and algebraic decision problems.

4. Gödel–Tarski–Löb (GTL) Consequences

The paper derives profound logical limitations for any formal system attempting to reason about exact capacity:

Gödel Incompleteness (Theorem VI.2): No sound, recursively axiomatizable theory (extending basic arithmetic) can be complete for exact capacity threshold statements. There will always be true statements about capacity that the theory cannot prove.
Tarski Undefinability (Theorem VI.5): No such theory can define its own truth predicate for capacity statements. Semantic truth strictly exceeds syntactic derivability.
Löb's Obstruction (Theorem VI.7): No consistent theory can prove the uniform reflection principle for capacity thresholds (i.e., it cannot prove that "if the theory proves $C_{fb} \geq q$ , then $C_{fb} \geq q$ " holds uniformly for all $e, q$ ).

4. Significance and Implications

Fundamental Limitation: The paper establishes a rigorous boundary between "structured" FSC subclasses (where exact formulas exist, e.g., for specific channels like the POST channel) and the general class of unifilar FSCs. It proves that the difficulty is not merely a lack of current technique but a fundamental computational barrier.
Clarification of "Exactness": It highlights that the "exact" infinite-horizon capacity is structurally distinct from finite-horizon approximations. While finite-horizon values are computable, their limit is not.
Methodological Shift: The results suggest that future progress in FSC feedback capacity must rely on:
1. Structural Assumptions: Focusing on specific subclasses with verifiable contraction or mixing properties.
2. Approximation Schemes: Accepting $\epsilon$ -approximations or promise-gap variants, which may remain decidable.
3. Meta-Theoretic Awareness: Recognizing that any attempt to create a "universal" exact solver for all FSCs is mathematically impossible.
Interdisciplinary Bridge: The paper uniquely bridges Shannon Information Theory with deep results in mathematical logic (Gödel, Tarski, Löb), demonstrating that information-theoretic quantities can encode undecidable arithmetic behaviors.

Conclusion

Angshul Majumdar's paper proves that the exact decision problem for feedback capacity in finite-state channels is undecidable. By constructing a "delayed-activation" channel family, the author demonstrates that finite-horizon observations are insufficient to determine asymptotic capacity. This leads to the conclusion that no universal exact algorithm, no finite-letter formula, and no complete formal proof system can exist for this problem. The work redefines the landscape of channel capacity research, shifting the focus from seeking universal exact solutions to identifying specific structural conditions where computability is restored.