Towards Tsallis Fully Probabilistic Design

1. Problem Statement

The paper addresses the challenge of extending Fully Probabilistic Design (FPD) to stochastic dynamic programming problems where the standard Kullback–Leibler (KL) divergence is replaced by the Tsallis divergence (TsD).

• Context: FPD is a framework for sequential Bayesian decision-making where the goal is to find a randomized policy (probability density function) that minimizes the divergence between the system's actual behavior and a desired reference behavior, subject to cost constraints.
• The Challenge: In standard FPD (using KL divergence), the chain rule allows for a tractable, closed-form backwards induction (dynamic programming) from the final stage $N$ to the initial stage $1$.
• The Obstacle: Tsallis divergence is non-additive. It satisfies a sub-additivity property (Lemma 2.4) rather than the standard chain rule. Consequently, the terms in the objective function cannot be separated cleanly by stage. A direct application of backwards induction fails because the optimization at stage $k$ depends on the specific densities of all other stages in a coupled, non-linear way.
• Goal: To establish the existence of a solution for the multistage FPD problem under Tsallis divergence and to provide a constructive algorithm to compute the optimal policy.

2. Methodology

The authors propose a double-loop fixed-point iteration scheme to overcome the lack of a direct chain rule.

A. Mathematical Foundations

• Tsallis Divergence: Defined using a deformed logarithm ($\log_r$) and exponential ($\exp_r$) with a shape parameter $r$. As $r \to 1$, it converges to KL divergence.
• Non-Additivity: The joint divergence $D_r(p(v,z) || q(v,z))$ includes a cross-term $(1-r)D_r(p(v|z)||q(v))D_r(p(z)||q(z))$, which couples the stages.
• Single-Stage Solution: The paper relies on Lemma 2.6 (from prior literature), which provides a closed-form solution for a single-stage Tsallis-regularized optimization problem involving a convex cost function.

B. The Iterative Scheme

Since direct backwards induction is impossible, the authors decompose the problem into a fixed-point iteration:

1. Operator $T$ (Inner Loop - Gauss-Seidel Backwards Induction):

   • Given a current estimate of the policy sequence $\{p^l_k\}$, the operator $T$ performs a backwards pass from $k=N$ to $k=1$.
   • At each stage $k$, it solves for the optimal policy $p^*_k$ while fixing the policies for all other stages ($j \neq k$) to their current estimates (either from the previous outer iteration $l$ or the current inner iteration $l+$).
   • This transforms the intractable multistage problem into a sequence of tractable single-stage problems (matching the structure of Lemma 2.6).
   • The solution for stage $k$ takes the form of a "deformed exponential" kernel twisting the reference density.

2. Operator $\tilde{T}_\omega$ (Outer Loop - Relaxation):

   • To ensure convergence, the authors define a relaxed operator: $\tilde{T}_\omega = \omega T + (1-\omega)I$, where $I$ is the identity and $0 < \omega \leq 1$.
   • This creates a standard fixed-point iteration: $p^{l+1} = \tilde{T}_\omega(p^l)$.
   • If $\omega = 1$, it is a pure Gauss-Seidel update; if $\omega < 1$, it is a damped update to prevent oscillation.

3. Key Contributions

1. Formulation of Tsallis FPD: The paper presents the first rigorous formulation of Fully Probabilistic Design using Tsallis divergence, offering greater flexibility in modeling systems with long-range correlations and fat-tail risks (where KL divergence may be insufficient).
2. Constructive Existence Proof: Unlike previous works that used Variational Inference to approximate solutions, this paper provides a constructive proof of the existence of an exact solution.
3. Algorithmic Scheme: The authors derive a specific double-loop fixed-point algorithm:
   • An inner loop performing a modified backwards induction (solving single-stage subproblems with fixed neighbors).
   • An outer loop performing fixed-point iteration with relaxation.
4. Convergence Guarantees: The paper establishes theoretical convergence based on standard theorems in Nonlinear Functional Analysis (Banach and Schauder Fixed Point Theorems).

4. Key Results and Theoretical Guarantees

The paper proves that the proposed iteration converges to the optimal policy under specific conditions:

• Contraction Mapping (Theorem 5.3): If the relaxation parameter is chosen such that $\omega < 1/2$, the operator $\tilde{T}_\omega$ is a contraction in the $L_1$ norm. By the Banach Fixed Point Theorem, this guarantees a unique fixed point and convergence from any starting density.
• Compactness and Stability (Theorem 5.4): Under stronger conditions on the cost function (e.g., quadratic growth $c_k(x) \geq \bar{c} + \|x\|^2$) or the Tsallis parameter ($r > 3$), the operator $T$ (with $\omega=1$) is shown to have a fixed point. By the Schauder Fixed Point Theorem, if the initial guess is sufficiently close to the solution, the iteration converges.
• Closed-Form Sub-solutions: The paper derives explicit formulas (Theorems 5.1 and 5.2) for the single-stage updates within the inner loop, showing they retain the structure of a deformed exponential distribution.

5. Significance

• Versatility in Decision Making: By moving beyond KL divergence, this framework allows for probabilistic modeling of systems where "tail risk" is critical (e.g., robotics, safety-critical control). Tsallis entropy naturally models long-range dependencies and fat tails, which KL-based models often fail to capture.
• Exact vs. Approximate: Previous applications of Tsallis divergence in Model Predictive Control (MPC) relied on variational approximations. This paper provides a path to the exact solution of the dynamic programming problem, ensuring theoretical guarantees that approximations cannot offer.
• Theoretical Bridge: It successfully bridges the gap between the elegant, tractable nature of KL-based FPD and the complex, non-additive nature of Tsallis statistics, demonstrating that fixed-point methods can restore tractability to non-additive stochastic control problems.

In summary, the paper successfully generalizes Fully Probabilistic Design to the Tsallis domain, replacing the standard backwards induction with a convergent fixed-point iteration, thereby enabling robust control strategies for systems with complex, non-Gaussian uncertainty structures.