Technical Summary: PL-KKT-hPINN

Problem Statement

Physics-informed neural networks (PINNs) have emerged as a powerful tool for process modeling, integrating governing physical equations into the loss function as soft penalties. However, a critical limitation of standard PINNs is that they do not guarantee exact satisfaction of physical laws during inference. The reliance on penalty terms often leads to an imbalance in the optimization landscape, resulting in slow convergence, sensitivity to hyperparameter tuning, and, most importantly, a failure to strictly enforce constraints. This is particularly problematic in optimization and control applications where the optimizer may explore poorly sampled regions, leading to physically infeasible solutions.

While "hard-constrained" neural networks exist to enforce constraints exactly, many existing approaches suffer from significant drawbacks:

• Iterative Solvers: Methods like OptNet or HardNet-Cvx rely on solving optimization problems (e.g., quadratic programs) within the network architecture, often requiring iterative fixed-point solvers that incur high computational costs and limit scalability.
• Non-Trainable Layers: Previous work by Chen et al. introduced KKT-hPINN, which enforces linear equality constraints via a non-trainable orthogonal projection layer derived from Karush–Kuhn–Tucker (KKT) conditions. This approach is computationally efficient and guarantees constraint satisfaction but is restricted to linear relationships.
• Nonlinearity Gap: Most chemical engineering relationships (e.g., species balances, kinetics, enthalpy balances) are inherently nonlinear. Existing methods for nonlinear constraints (e.g., Picard-KKT-hPINN, Adaptive-depth Neural Projection, KKT-HardNet) often rely on sequential, iterative projection procedures or variable-freezing strategies, which increase computational overhead and prevent the use of static, non-iterative architectures.

Methodology: PL-KKT-hPINN

The authors propose Piecewise-Linear Karush–Kuhn–Tucker hard-constrained PINNs (PL-KKT-hPINN), a framework designed to strictly enforce nonlinear equality constraints while maintaining the computational efficiency and static architecture of the original KKT-hPINN.

Core Concept

The method approximates nonlinear equality constraints $g(x, y) = 0$ locally over multiple regions of the input domain using piecewise-linear equality constraints. Instead of solving a complex nonlinear projection problem, the framework constructs a set of local linear approximations, each valid within a specific subregion $R_j$.

Mathematical Formulation

1. Piecewise-Linear Approximation: The input domain is partitioned into $p$ non-overlapping subregions $R_j$. Within each region, the nonlinear constraint is approximated via a first-order Taylor expansion around a representative point $(x^c_j, y^c_j)$ from the training data:
   $$g_i(x, y) \approx g_i(x^c_j, y^c_j) + \nabla_x g_i(x^c_j, y^c_j)^T (x - x^c_j) + \nabla_y g_i(x^c_j, y^c_j)^T (y - y^c_j)$$
   This yields a local linear constraint of the form $A_j x + B_j y = b_j$.

2. Local Projection: For each region $R_j$, the unconstrained neural network prediction $\hat{y}$ is projected onto the local feasible hyperplane defined by $A_j x + B_j y = b_j$. This projection is computed using a closed-form analytic solution derived from KKT conditions (similar to the linear KKT-hPINN):
   $$\tilde{y}_j = A^*_j x + B^*_j \hat{y} + b^*_j$$
   where $A^*_j, B^*_j, b^*_j$ are fixed, non-trainable matrices calculated once based on the local linearization.

3. Aggregation via Indicator Functions: The final constrained prediction $\tilde{y}$ is obtained by selecting the active regional projection based on the input $x$. This is achieved using an indicator function $\Omega_{R_j}(x)$ (implemented via Heaviside step functions) which is 1 if $x \in R_j$ and 0 otherwise:
   $$\tilde{y} = \sum_{j=1}^{p} \Omega_{R_j}(x) \tilde{y}_j$$

Architecture and Training

• Static Architecture: The projection layers are non-trainable and appended to the neural network. The architecture remains static during both training and inference, avoiding the need for iterative solvers.
• Parallel Execution: Unlike sequential projection methods, PL-KKT-hPINN computes all regional projections in parallel and aggregates them, enabling efficient vectorized computation.
• Loss Function: The model is trained by minimizing the mean squared error between the final projected prediction $\tilde{y}$ and the ground truth $y$. Since the indicator functions depend only on the fixed input $x$ and not on trainable parameters, the loss function remains differentiable with respect to the network weights.

Key Contributions

1. Extension to Nonlinear Constraints: The paper extends the KKT-hPINN framework, previously limited to linear constraints, to handle general nonlinear equality constraints through piecewise-linear approximation.
2. Non-Iterative Efficiency: The method provides a computationally efficient, non-iterative alternative to existing hard-constrained approaches for nonlinear systems. It eliminates the need for solving optimization problems at every forward pass.
3. Static and Parallelizable: The architecture is static and amenable to parallelization, addressing the scalability issues found in iterative projection methods.
4. Quantifiable Approximation Error: The only source of error introduced by the projection is the piecewise-linear approximation, which can be quantified a priori and reduced by increasing the number of linearization regions.

Experimental Results

The framework was evaluated on a steady-state Continuous Stirred-Tank Reactor (CSTR) case study involving reversible reactions with nonlinear kinetics (Arrhenius equations). Two scenarios were tested: a 1D input case (varying feed concentration) and a 2D input case (varying feed concentration and temperature).

• Constraint Satisfaction: PL-KKT-hPINN achieved substantially lower constraint violations compared to standard Neural Networks (NNs) and soft-constrained PINNs. The violations were reduced to the magnitude of the piecewise-linear approximation error (orders of magnitude lower than PINNs).
• Predictive Accuracy: The method preserved predictive accuracy comparable to standard NNs, with Root Mean Square Error (RMSE) values nearly identical to unconstrained models.
• Data Efficiency: In low-data regimes, PL-KKT-hPINN demonstrated improved robustness and lower RMSE than unconstrained NNs. The structural regularization provided by the projection layer helped prevent overfitting when training data was limited.
• Scalability and Cost: Increasing the number of linearization regions systematically improved constraint satisfaction. While computational cost at inference increased linearly with the number of regions, the method remained significantly more efficient than iterative approaches. The 2D case showed slightly higher violations due to the high sensitivity of Arrhenius terms to temperature, highlighting the trade-off between approximation accuracy and region granularity.

Significance and Claims

The authors claim that PL-KKT-hPINN provides a computationally efficient and physically consistent framework for the surrogate modeling of nonlinear chemical engineering systems.

• Guaranteed Feasibility: Unlike PINNs, the method guarantees constraint satisfaction (up to the piecewise-linear approximation error) during both training and inference.
• No Hyperparameter Tuning: The framework avoids the challenging task of tuning penalty weights required by PINNs to balance loss terms.
• Practical Applicability: By maintaining a static architecture and avoiding iterative solvers, the method is suitable for real-time applications and optimization loops where computational speed and reliability are critical.

The paper concludes that while the approximation error depends on the number of linearization regions, the method offers a systematic way to enforce nonlinear constraints without sacrificing the efficiency of closed-form projections, making it a promising tool for industrial process modeling.