Adaptive Tuning of Online Feedback Optimization for Process Control Applications

1. Problem Statement

Online Feedback Optimization (OFO) is a control strategy used to drive systems to optimal operating points in real-time using limited model information (specifically, input-output sensitivity rather than full process models). It is particularly valuable in process control where accurate dynamic models are difficult to obtain.

However, OFO controllers based on scaled projected gradient descent rely heavily on two hyperparameters:

1. Step size ($\alpha$): Determines the magnitude of the update at each iteration.
2. Scaling matrix ($S$): A metric that shapes the gradient direction and magnitude.

The Core Challenge:

• Disconnection: These parameters are disconnected from the physical characteristics of the underlying process.
• Tuning Difficulty: Traditionally, tuning these parameters requires repeated experiments or extensive prior knowledge of the system to find values that ensure stability and fast convergence.
• Limitations of Existing Methods: Existing adaptive methods often assume unconstrained problems, linear dynamics, or require Hessian information (which is unavailable in OFO). Furthermore, standard line-search strategies are not applicable due to the reliance on real-time measurements.
• Goal: The paper aims to develop an adaptive tuning method that automatically adjusts $\alpha$ and $S$ online, eliminating the need for repeated experiments and reducing operator-tunable parameters to simple scalar values.

2. Methodology

The proposed method leverages the sensitivity of the objective function with respect to the tuning parameters ($S$ and $\alpha$). Instead of running new experiments to estimate these sensitivities, the authors derive them mathematically using the chain rule and the known input-output sensitivity ($\nabla h$) already utilized by standard OFO controllers.

A. Scaling Matrix Adaptation ($S$)

The goal is to ensure the objective function $\Phi$ decreases monotonically between iterations.

• Approximation: Since the full input-output map $h$ is unknown, the method approximates the next state using the linear sensitivity: $y_{k+1} \approx y_k + \alpha_{max} \nabla h(u_k) w_k$.
• Optimization Formulation: The method formulates an auxiliary semi-definite optimization problem to find the next scaling matrix $S_{k+1}$.
  • Objective: Maximize the decrease in the objective function (minimize $-p$).
  • Constraints:
    • Ensure the inner product of the sensitivity gradient and the change in $S$ is negative (guaranteeing descent).
    • Maintain $S$ as a symmetric positive definite matrix (via eigenvalue constraints).
    • Limit the spectral norm of $S$ to prevent excessive changes.
• Diagonal Heuristic: To avoid solving the full optimization problem at every step, a simplified diagonal adaptation rule is proposed. It adjusts diagonal elements of $S$ based on the sign of the sensitivity derivative:
  • If $\frac{\partial \Phi}{\partial S_i} < 0$, increase $S_i$.
  • If $\frac{\partial \Phi}{\partial S_i} > 0$, decrease $S_i$.
  • This resembles a two-way backtracking algorithm.

B. Step Size Adaptation ($\alpha$)

To mitigate inaccuracies in the linear approximation of the system dynamics:

• Quadratic Approximation: The objective function is approximated as a quadratic function of the step size $\alpha$: $g(\alpha) = a_k\alpha^2 + b_k\alpha + c_k$.
• Coefficients: The coefficients are derived using the current objective value, the gradient, and a predicted value at a trial step size.
• Optimization: The optimal step size $\alpha_{k+1}$ is found by minimizing this quadratic approximation within a bounded range $[\alpha_{min}, \alpha_{max}]$.

C. Algorithm Structure

The algorithm iterates as follows:

1. Check if the optimum is reached (gradient is zero).
2. If not, solve the auxiliary optimization (or use the heuristic) to update $S_k$.
3. Solve the projected gradient step to find the search direction $w_k$.
4. Solve the quadratic approximation to find the optimal step size $\alpha_k$.
5. Update the control input $u_{k+1}$ and measure the new output $y_{k+1}$.

3. Key Contributions

1. Adaptive Tuning Framework: A novel method to adapt both the scaling matrix and step size online without requiring repeated system experiments.
2. Sensitivity-Based Derivation: Derivation of sensitivity formulas for the objective function with respect to $S$ and $\alpha$, utilizing only the input-output sensitivity ($\nabla h$) and standard OFO measurements.
3. Reduced Operator Burden: Reduces the tuning problem to selecting scalar bounds ($p_{max}, t_{min}, t_{max}, \alpha_{min}, \alpha_{max}$) rather than tuning complex matrix parameters.
4. Theoretical Guarantees (Approximate): The method ensures monotonic descent of the objective function under the linear approximation of the system dynamics.

4. Results

The method was validated through numerical studies on three distinct cases:

• Case 1: Non-Convex Toy Example (Haberle et al. benchmark):

  • The adaptive scheme reached the optimum in 5 iterations, compared to 82 iterations for a fixed step size and identity scaling matrix.
  • It successfully respected non-convex feasible regions.

• Case 2: Rosenbrock Function:

  • Demonstrated that combining scaling matrix adaptation with step size adaptation prevents oscillations common in "valley" functions.
  • Fixed scaling with step adaptation reduced oscillations but was slower; fixed step with scaling adaptation was unstable. The combined approach yielded fast convergence without oscillations.

• Case 3: Gas Lift Optimization (Industrial Application):

  • Applied to a system with two oil platforms and five wells.
  • The adaptive method (both full optimization and diagonal heuristic) converged significantly faster than a manually tuned fixed-step approach.
  • The heuristic approach required fewer iterations to adapt than the full optimization in the early stages but both outperformed manual tuning.

• Case 4: Continuously Stirred Tank Reactor (CSTR) - Dynamic System:

  • Compared against manual tuning (which required 8 experimental cases to find a good configuration).
  • Performance: The adaptive algorithm achieved 25% better tracking accuracy on the tuning trajectory and 20% better on a validation trajectory compared to the best manually tuned parameters.
  • Efficiency: Achieved this improvement without any repeated experiments, whereas manual tuning required extensive trial-and-error.

5. Significance and Conclusion

This paper addresses a critical bottleneck in the practical deployment of Online Feedback Optimization: the difficulty of tuning hyperparameters for complex, constrained, nonlinear processes.

• Practical Impact: By automating the tuning process, the method makes OFO more accessible to process engineers who may lack deep expertise in optimization theory or the time to conduct extensive tuning experiments.
• Robustness: The ability to adapt to changing operating conditions (e.g., constraint activation) in real-time makes the controller more robust than static, manually tuned controllers.
• Future Work: The authors note that while the method performs well empirically, future research is needed to provide rigorous theoretical guarantees for convergence and constraint satisfaction, particularly in the presence of noisy real-world measurements.

In summary, the proposed adaptive tuning scheme transforms OFO from a method requiring significant manual intervention into a self-tuning controller capable of rapid, stable convergence in complex process control environments.