Lecture Note for Bounded Controls in Continuous-Time and Control of Several Variables

1. Problem Statement and Motivation

The notes address optimal control problems where the control variable $u(t)$ is subject to box constraints (pointwise bounds), a common scenario in biological, pharmacological, economic, and engineering systems (e.g., dosage limits, actuator saturation).

The core problem is formulated as a Bolza problem:
$$\max_{u(\cdot)} J(u) = \int_{t_0}^{t_1} f(t, x(t), u(t)) \, dt + \phi(x(t_1))$$
Subject to:
$$\dot{x}(t) = g(t, x(t), u(t)), \quad x(t_0) = x_0, \quad u(t) \in U = [a_1, b_1] \times \dots \times [a_m, b_m]$$

Key Challenge: The notes emphasize a critical misconception in applied control: one cannot simply solve the unconstrained optimality system and then "clip" (truncate) the resulting control to fit the bounds. Because the state $x$ and adjoint $\lambda$ are coupled to the control $u$, the constraints must be enforced within the optimality system itself.

2. Methodology and Theoretical Framework

The notes develop a first-order theory based on Pontryagin's Maximum Principle (PMP) adapted for compact control sets.

A. Hamiltonian Formalism with Constraints

• Hamiltonian Definition: $H(t, x, u, \lambda) = f(t, x, u) + \lambda^\top g(t, x, u)$.
• Constrained PMP: Instead of the unconstrained stationarity condition $\frac{\partial H}{\partial u} = 0$, the optimal control $u^*(t)$ must maximize the Hamiltonian pointwise over the admissible set $U$:
  $$u^*(t) \in \arg\max_{v \in U} H(t, x^*(t), v, \lambda^*(t))$$
• Adjoint Equations: The state and costate equations remain $\dot{x} = H_\lambda$ and $\dot{\lambda} = -H_x$, but the boundary conditions for $\lambda$ depend on the terminal constraints.

B. Characterization of the Optimal Control

The notes derive specific structural forms for the control law based on the Hamiltonian's dependence on $u$:

1. Scalar Box Constraints (Sign Conditions):
   For a scalar control $u \in [a, b]$, the optimality condition is equivalent to a set of complementary sign conditions:

   • If $H_u < 0 \implies u^* = a$ (Lower bound).
   • If $H_u > 0 \implies u^* = b$ (Upper bound).
   • If $H_u = 0 \implies a < u^* < b$ (Interior).
   • Variational Derivation: Proven via one-sided perturbations, showing that the gradient must point outward from the feasible set at the boundary.

2. Quadratic Hamiltonians (Projection Formula):
   For models where $H$ is strictly concave/convex and quadratic in $u$ (e.g., $H = \Psi + \beta u - \frac{\rho}{2}u^2$), the unconstrained maximizer is $\tilde{u} = \beta/\rho$. The constrained solution is the exact projection (clamping) of this unconstrained value onto the interval:
   $$u^*(t) = \text{clamp}_{[a,b]}(\tilde{u}(t)) = \min\{b, \max\{a, \tilde{u}(t)\}\}$$
   Crucial Insight: This projection is not a post-hoc numerical fix; it is the analytical solution to the constrained maximization step.

3. Affine Hamiltonians (Bang-Bang Control):
   If $H$ is linear in $u$ ($H = \Psi + \sigma(t)u$), the control is bang-bang:
   $$u^*(t) = \begin{cases} b & \text{if } \sigma(t) > 0 \\ a & \text{if } \sigma(t) < 0 \end{cases}$$
   Switching occurs when the switching function $\sigma(t)$ crosses zero. Singular arcs (where $\sigma(t) \equiv 0$) require higher-order analysis.

4. Multi-Input and Normal Cones:
   For vector controls $u \in \mathbb{R}^m$, the condition is expressed via the normal cone $N_U(u^*)$:
   $$0 \in -H_u(t, x^*, u^*, \lambda^*) + N_U(u^*)$$
   This generalizes the sign conditions to multi-dimensional box constraints, decoupling component-wise if the Hamiltonian is separable.

C. Numerical Implementation: Forward-Backward Sweep (FBSM)

The notes detail the Forward-Backward Sweep Method for solving these problems numerically:

1. Forward: Integrate state equations with current control guess.
2. Backward: Integrate adjoint equations with terminal conditions.
3. Update: Compute the new control by pointwise maximizing the Hamiltonian (applying the projection/clamping formula) using the current $x$ and $\lambda$.
   • Correction: The update must be $u^{(k+1)} = \text{clamp}(\Gamma(t, x^{(k)}, \lambda^{(k)}))$, not a clipping of a previously unconstrained trajectory.

3. Key Contributions and Extensions

The notes extend the theory to several advanced topics:

• Multi-State/Multi-Control Systems: Generalizes the PMP to $n$ states and $m$ controls, defining the componentwise adjoint system and transversality conditions.
• Problem Reduction:
  • Structural Elimination: Removing state variables that do not affect the objective or dynamics of other states.
  • Explicit Integration: Converting integral objectives or constraints into state variables.
• Finite-Horizon LQR (Linear Quadratic Regulator):
  • Derives the Riccati Differential Equation for the feedback gain matrix $S(t)$.
  • Shows that for linear systems with quadratic costs, the optimal control is a linear state feedback $u^* = -R^{-1}B^\top S(t)x$, even with bounds (though bounds complicate the closed-form solution, the unconstrained LQR structure is the baseline).
• Higher-Order Systems: Demonstrates how to convert $n$-th order differential equations into first-order state-space systems to apply PMP.
• Isoperimetric Constraints:
  • Shows how to convert integral constraints (e.g., $\int h(t, x, u) dt = B$) into endpoint constraints by introducing an auxiliary state variable $z(t)$ with $\dot{z} = h$. This allows standard PMP application without external Lagrange multipliers.

4. Results and Significance

Key Results:

1. The Projection Principle: For quadratic Hamiltonians, the constrained optimal control is analytically the projection of the unconstrained control onto the feasible set.
2. The "Clipping" Fallacy: The notes rigorously prove that solving the unconstrained system and then truncating the result is generally incorrect because it fails to satisfy the coupled state-adjoint boundary value problem required by the constraints.
3. Structural Decomposition: The optimal control trajectory often exhibits a bound-interior-bound structure (saturation, interior arc, decay), which must be captured by the numerical solver.

Significance:

• Pedagogical Clarity: The notes bridge the gap between abstract variational analysis and practical implementation, specifically targeting senior undergraduates and PhD students in applied mathematics.
• Biological Modeling: By referencing Lenhart and Workman, the notes provide a robust framework for biological models (e.g., epidemiology, pharmacokinetics) where control inputs (drug dosage, harvesting) are physically bounded.
• Algorithmic Correctness: The emphasis on enforcing constraints during the Forward-Backward Sweep iteration is critical for the convergence and accuracy of numerical solvers in nonlinear optimal control.
• Unification: The notes unify various control scenarios (bang-bang, singular, LQR, isoperimetric) under a single Hamiltonian framework, clarifying the geometric interpretation of optimality via normal cones.

In summary, this lecture note serves as a comprehensive technical guide for handling bounded controls in continuous-time systems, correcting common misconceptions about post-hoc truncation and providing the rigorous mathematical machinery (PMP, projection formulas, Riccati equations) necessary for solving complex multi-variable control problems.