Thermodynamically consistent machine learning model for excess Gibbs energy

1. Problem Statement

The accurate prediction of the molar excess Gibbs energy ($g^E$) and derived activity coefficients ($\gamma_i$) is fundamental to chemical engineering for modeling phase equilibria (Vapor-Liquid Equilibrium - VLE, Liquid-Liquid Equilibrium - LLE) and reaction kinetics.

• Current Limitations:
  • Physics-based models (NRTL, UNIQUAC): Require extensive experimental data to fit binary interaction parameters. They cannot predict properties for unstudied components or systems lacking binary data. They often struggle to simultaneously describe VLE and LLE with a single parameter set.
  • Group Contribution (UNIFAC): Reduces parameters by decomposing molecules into functional groups but is limited to pre-parameterized groups and struggles with complex systems (e.g., ionic liquids) or simultaneous VLE/LLE description.
  • Quantum Chemical (COSMO-RS/SAC): Computationally expensive and accuracy depends heavily on the chosen DFT level.
  • Existing ML Models: Many recent Graph Neural Network (GNN) approaches lack strict thermodynamic consistency (e.g., violating the Gibbs-Duhem equation) or are restricted to specific conditions (e.g., infinite dilution only, fixed temperature, or binary systems only).

The Core Challenge: Developing a flexible machine learning model that predicts $g^E$ and activity coefficients for arbitrary mixtures (including unstudied components) solely from molecular structures, while strictly adhering to all thermodynamic consistency laws.

2. Methodology: The HANNA Model

The authors introduce HANNA (Hybrid Architecture for Neural Network Activity), a deep learning framework that integrates physical laws as hard constraints directly into the neural network architecture.

A. Architecture and Input

• Inputs: Molecular structures (SMILES strings), temperature ($T$), and mole fractions ($x$).
• Embedding: SMILES strings are processed using the pre-trained transformer model ChemBERTa-2 to generate molecular embeddings.
• Network Structure: HANNA utilizes three feed-forward neural networks (FFNNs):
  1. Embedding Network: Refines molecular embeddings.
  2. Mixture Network: Processes component interactions.
  3. Property Network: Computes binary interaction terms.
• Hard Constraints: The architecture enforces:
  • Permutation Equivariance: Output order matches input component order.
  • Pure Component Limit: Activity coefficients approach 1 for pure components.
  • Gibbs-Duhem Consistency: Activity coefficients are derived via automatic differentiation of the predicted $g^E$, ensuring mathematical consistency.
  • Identical Component Handling: A radial basis function (RBF) similarity score lumps identical components to ensure $g^E = 0$ for pure substances.

B. Multi-Component Extrapolation

To extend from binary training data to multi-component mixtures without additional parameters, HANNA employs a geometric projection method (Muggianu projection).

• The mole fractions of an $N$-component mixture are projected onto all $N(N-1)/2$ binary subsystems.
• The total $g^E$ is calculated as a weighted sum of the binary subsystem contributions.
• This allows the model to be trained exclusively on binary data while predicting ternary and quaternary systems.

C. Training Strategy & Loss Function

HANNA is trained end-to-end on a massive dataset from the Dortmund Data Bank (DDB) covering 46,543 systems. The training minimizes a composite loss function ($L_{total}$):

1. Data Losses:
   • VLE (TPXY/TPX): Predicts activity coefficients or total pressure.
   • Infinite Dilution (ACI): Predicts $\gamma^\infty$.
   • Excess Enthalpy ($H^E$): Derived via automatic differentiation of $g^E$ with respect to temperature.
   • LLE (Liquid-Liquid Equilibrium): This is a key innovation. Since LLE requires solving non-linear equations iteratively (which breaks gradient flow), the authors developed a differentiable surrogate solver. This solver uses a neural network to mimic the Convex Envelope Method (CEM), mapping the mixing Gibbs energy ($\Delta g_{mix}$) directly to phase compositions ($x', x''$).
2. Gibbs Loss ($L_{Gibbs}$): A physics-inspired loss term that penalizes the model if it fails to predict a miscibility gap (i.e., if the second derivative of $\Delta g_{mix}$ is not negative where required).
3. Lipschitz Regularization ($L_{Lips}$): Enforces smoothness of the predicted $g^E$ function to improve robustness and extrapolation capabilities.

3. Key Contributions

1. Thermodynamic Consistency as Hard Constraints: HANNA is the first model to strictly embed all relevant thermodynamic constraints (Gibbs-Duhem, infinite dilution limits, permutation symmetry) into the network architecture rather than relying on soft penalties.
2. End-to-End LLE Training: The development of a differentiable surrogate solver allows for the direct training of ML models on experimental LLE phase composition data, a task previously considered difficult for deep learning due to the need for iterative solvers.
3. Universal Applicability: Unlike UNIFAC, HANNA requires no group parameterization. It can predict properties for any mixture where molecular structures are known, including ionic liquids and complex systems.
4. Simultaneous VLE/LLE Description: The model successfully describes both vapor-liquid and liquid-liquid equilibria with a single parameterization, overcoming a major limitation of traditional models like UNIQUAC.
5. Open Source: The model, code, and an interactive web interface (MLPROP) are publicly available.

4. Results

The model was evaluated on unseen test systems (withheld during training) and compared against modified UNIFAC (Dortmund) and other ML benchmarks (GE-GNN, GDI-GNN, SolvGNN).

• Accuracy vs. UNIFAC:
  • Binary Systems: HANNA outperforms modified UNIFAC in Mean Absolute Error (MAE) for VLE, ACI, and LLE. Notably, the median MAE for ACI dropped from 0.18 (UNIFAC) to 0.11 (HANNA).
  • Multi-Component Systems: HANNA maintains superior or comparable accuracy for ternary mixtures.
  • LLE Detection: HANNA correctly identifies miscibility gaps in 89% of binary LLE systems (vs. 74% for UNIFAC) and 98% of ternary LLE systems.
• Applicability:
  • HANNA successfully predicts properties for systems containing ionic liquids and complex molecules where UNIFAC fails due to missing group parameters.
  • It covers a broader chemical space, including unstudied components.
• Comparison with other ML models:
  • HANNA significantly outperforms other ML models (GE-GNN, SolvGNN, etc.) in accuracy and applicability.
  • Unlike other ML models restricted to binary systems, fixed temperatures, or infinite dilution, HANNA handles arbitrary $N$-component mixtures and temperature/composition dependence.
• Qualitative Performance: The model correctly predicts complex phase behaviors, including island-type miscibility gaps (upper and lower critical solution temperatures) and heteroazeotropes.

5. Significance and Impact

• Paradigm Shift: HANNA represents a shift from "fitting parameters to data" to "learning physical laws from data." It demonstrates that neural networks can be constrained to obey thermodynamics without sacrificing flexibility.
• Industrial Utility: By providing accurate predictions for unstudied systems (e.g., new solvents, ionic liquids) without requiring new experimental fitting, HANNA accelerates process design, solvent selection, and material discovery.
• Methodological Advancement: The surrogate solver for LLE and the geometric projection strategy offer new tools for integrating complex physical constraints into deep learning workflows.
• Future Outlook: The authors plan to extend the training to Solid-Liquid Equilibria (SLE) and potentially incorporate ternary data directly to refine the projection method.

In summary, HANNA sets a new state-of-the-art for predictive thermodynamics, combining the generalization power of deep learning with the rigorous consistency of physical laws, outperforming decades-old industry standards while offering broader applicability.