Technical Summary: A Complete Characterisation of Conditional Entropies

Problem Statement

Conditional entropy extends the concept of uncertainty quantification to scenarios where an observer possesses side information ($Y$) correlated with a random variable ($X$). While various definitions of conditional Rényi entropies exist in the literature (e.g., those by Hayashi, Arimoto, Cachin, and Renner), a complete axiomatic characterization of the most general form of conditional entropy has remained elusive. Existing axiomatic approaches often rely on mathematical conveniences rather than operational justifications, failing to fully explain the role of Rényi entropies in information theory or to unify the disparate definitions of conditional entropy under a single framework.

The central problem addressed in this work is to derive a complete characterization of conditional entropy measures that satisfy a natural set of operational axioms:

1. Invariance under relabeling: Entropy should not change under outcome relabeling or embedding into larger alphabets.
2. Monotonicity under conditional mixing: Entropy should not decrease when a mixing (doubly stochastic) channel acts on the system $X$, potentially dependent on the side information $Y$.
3. Additivity: Entropy for independent random variables must be additive.
4. Normalization: The entropy of a fair coin toss is fixed to 1 bit.

Methodology

The authors employ a rigorous algebraic approach based on the theory of preordered semirings, specifically utilizing results from [16, 17] regarding large-sample transformations and catalytic ordering.

1. Semiring Construction: The authors construct a specific preordered semiring, the semiring of conditional majorization ($S$). The elements of this semiring are equivalence classes of joint probability distributions (matrices) under relabeling and embedding. The preorder is defined by the existence of a "conditionally mixing channel" (a specific class of linear maps involving doubly stochastic and sub-stochastic matrices) that transforms one distribution into another.
2. Large-Sample Theorem: They apply a "Vergleichsstellensatz" (comparison theorem) for preordered semirings (Theorem 3.1). This theorem states that the ability to transform multiple copies of one element into another (in the limit of large $n$) is determined by the values of monotone homomorphisms and derivations on the semiring.
3. Characterization of Homomorphisms and Derivations: The core technical effort involves explicitly deriving the general form of these monotone homomorphisms and derivations for the conditional majorization semiring.
   • Homomorphisms: These are shown to correspond to exponential averages of Rényi entropies of the conditioned distribution.
   • Derivations: These correspond to limiting cases (specifically as the parameter $t \to 0$) and represent linear combinations of conditional Rényi entropies.
4. Convexity Analysis: To ensure the derived measures satisfy the monotonicity axiom (monotonicity under conditionally mixing channels), the authors analyze the convexity and concavity properties of the resulting functions. They establish sufficient conditions on the parameters (a real parameter $t$ and a probability measure $\tau$) using Minkowski's inequality and properties of multivariate divergences. Necessary conditions are derived by constructing explicit counterexamples in high-dimensional spaces.

Key Contributions and Results

1. General Form of Conditional Entropy

The paper establishes that any conditional entropy satisfying the four operational axioms can be expressed as a generalized convex combination (barycentric decomposition) of a specific family of extremal conditional entropies.

The general form is given by:
$$H_{t, \tau}(X|Y) = \frac{1}{t} \log \sum_{y} P(y) \exp \left( t \int_{[0, \infty]} H_\alpha(X|Y=y) \, d\tau(\alpha) \right)$$
where:

• $t \in \mathbb{R} \setminus \{0\}$ is a real parameter.
• $\tau$ is a probability measure on the positive reals $[0, \infty]$.
• $H_\alpha$ denotes the standard Rényi entropy of order $\alpha$.

This family includes the limiting cases:

• Bulk entropies ($t \neq 0$): The general form above.
• Tropical limits ($t \to \pm \infty$): Corresponding to min/max operations over the side information outcomes.
• Derivations ($t \to 0$): Corresponding to linear averages, specifically $H_{0, \alpha}(X|Y) = \sum_y P(y) H_\alpha(X|Y=y)$.

2. Admissible Parameter Sets

The authors derive sufficient conditions on the parameters $(t, \tau)$ to ensure the resulting entropy is monotone under conditionally mixing channels. These conditions are:

• Case $t > 0$: $\text{supp}(\tau) \subseteq [0, 1]$ and $\int \frac{\alpha}{1-\alpha} d\tau(\alpha) \leq \frac{1}{t}$.
• Case $t < 0$: $\text{supp}(\tau) \subseteq [0, 1]$ OR $\text{supp}(\tau) \subseteq [0, 1] \cup \{\alpha^*\}$ (for some $\alpha^* > 1$) such that the integral condition holds.

The paper also establishes necessary conditions for these parameters under the assumption that the measure $\tau$ vanishes sufficiently fast as $\alpha \to 1$ (which includes all discrete measures with finite support). The authors conjecture that these sufficient conditions are necessary in full generality.

3. Operational Interpretations

• Transformation Rates: The conditional entropies $H_{t, \tau}$ determine the optimal asymptotic rate $R$ for transforming multiple copies of a joint distribution $P_{XY}$ into $Q_{X'Y'}$ under conditionally mixing channels. The rate is given by the infimum of the ratio of the entropies of the target to the source over all admissible parameters:
  $$R(P_{XY} \to Q_{X'Y'}) = \inf \left\{ \frac{H_{t, \tau}(X'|Y')_Q}{H_{t, \tau}(X|Y)_P} \right\}$$
• Second Laws of Thermodynamics: The framework is applied to derive a set of "second laws" for quantum thermodynamics with side information. For states diagonal in the energy eigenbasis, the authors define generalized free energies $F_{t, \tau}$ (constructed from $H_{t, \tau}$ and Rényi relative entropies). These quantities provide sufficient conditions for the existence of catalytic, conditionally Gibbs-preserving transformations.

Significance and Claims

The paper claims to provide the first complete characterization of conditional entropies based on a natural set of operational axioms. By unifying various existing definitions (such as Hayashi's, Arimoto's, and Cachin's entropies) as special cases of the derived general form, the work reveals an underlying structure common to these measures.

The significance lies in:

1. Unification: It demonstrates that the diverse landscape of conditional Rényi entropies is not arbitrary but stems from a single, parameterized family of exponential averages.
2. Operational Completeness: It links the axiomatic definition directly to the operational task of asymptotic state transformation, proving that the derived family is both necessary and sufficient for characterizing transformation rates.
3. Thermodynamic Application: It extends the "second laws" of quantum thermodynamics to scenarios with side information, showing that the presence of side information significantly complicates the set of necessary and sufficient conditions for state transformations, requiring a family of free energies rather than a single one.

The authors remain modest regarding the complete characterization of the parameter sets $D_{bulk}$ and $D_{-\infty}$, noting that while they have proven necessity for discrete measures and conjecture it for general measures, a full proof for arbitrary measures remains an open problem. Similarly, the results on thermodynamic transformations currently rely on the assumption of rational Gibbs spectra, which the authors identify as a limitation to be addressed in future work.