💻 computer science

Inducing Permutation Invariant Priors in Bayesian Optimization for Carbon Capture and Storage Applications

This paper addresses the inefficiency of standard Bayesian Optimization for problems with permutation symmetries, such as well placement in Carbon Capture and Storage, by introducing a novel permutation-invariant Gaussian Process kernel (GP-Perm) and a Deep Kernel Learning baseline (DKL-DS) that are validated across synthetic benchmarks and a realistic CCS case study.

Original authors: Sofianos Panagiotis Fotias, Vassilis Gaganis

Published 2026-05-22

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sofianos Panagiotis Fotias, Vassilis Gaganis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find the perfect layout for a garden to grow the most vegetables possible. You have a limited number of seeds (budget) to plant, and every time you plant a new arrangement, you have to wait a whole year to see how well it grows (this is the "expensive simulation").

This is the challenge faced by scientists trying to design Carbon Capture and Storage (CCS) projects. They need to figure out exactly where to drill injection wells (to pump CO2 underground) and production wells (to manage pressure) to store the most carbon safely. But running a computer simulation to test one layout takes a huge amount of computing power and time.

The Problem: The "Name Tag" Confusion

The researchers noticed a specific problem with how computers usually learn these layouts.

Imagine you have a group of 5 friends (injector wells) and 3 other friends (producer wells). If you ask a standard computer program, "How good is this group of friends?" it treats the order in which you list them as important.

Scenario A: You list them as [Alice, Bob, Charlie, Dave, Eve].
Scenario B: You list them as [Eve, Dave, Charlie, Bob, Alice].

To a human, these are the exact same group of friends doing the exact same thing. But to a standard computer model, these look like two completely different groups. The computer wastes its limited "seeds" (simulations) trying to learn that Alice is the same as Eve, Bob is the same as Dave, and so on. It gets confused and inefficient.

In the paper, the authors call this Permutation Invariance. It means the result shouldn't change just because you shuffled the order of the items in the list.

The Solution: The "Smart Garden Planner" (GP-Perm)

The authors created a new tool called GP-Perm. Think of this as a super-smart garden planner that understands the concept of a "set" rather than a "list."

Instead of looking at the order of the wells, GP-Perm looks at the shape and relationships of the group.

It uses a special mathematical trick called Sinkhorn divergence. Imagine this as a way to measure how "far apart" two groups of friends are, regardless of who is standing where in the line. It checks the distances between everyone in the group, not just their positions in a list.
It also looks at how the "injector" group and the "producer" group interact with each other, like checking if the water pipes connect well, without caring about the order they were listed.

By ignoring the irrelevant "name tags" (the order), the computer can learn much faster. It realizes, "Oh, I already know this layout works, even if you listed the wells in a different order!"

The Competition: Learning vs. Rules

To make sure their new planner was the best, the authors compared it against other methods:

The "Clueless" Planner (Standard GP): Treats every order as unique. It learns slowly and gets confused easily.
The "Deep Learner" (DKL-DS): This is a neural network that tries to learn the rule that order doesn't matter. It's like a student trying to memorize that "Alice = Eve" by studying many examples. The problem is, with very few examples (which is the case in CCS), the student might get confused or "drift" and forget what it learned.
The "Old School" Planners (Set Kernels): These use older math tricks to ignore order. They are good, but sometimes they smooth over the fine details of the garden layout too much.

The Results: Faster and Smarter

The authors tested their new planner on two types of challenges:

Synthetic Tests: Fake, made-up problems designed to test the logic.
Real-World Test: A simulation of the Johansen formation, a real geological site in Norway where carbon storage is being studied.

The findings were clear:

The GP-Perm planner consistently found better well layouts using fewer simulations than the "Clueless" planner.
It was more reliable and stable than the "Deep Learner," which sometimes got confused when data was scarce.
It performed as well as or better than the "Old School" planners, but with a better understanding of the specific geometry of the wells.

The Takeaway

In the world of expensive, slow experiments like designing carbon storage, you can't afford to waste time on things that don't matter. By teaching the computer that shuffling the order of the wells doesn't change the physics, the authors built a tool that finds the best designs faster, safer, and more efficiently.

They didn't just invent a new math formula; they built a smarter way to ask the computer questions, ensuring it focuses only on the geometry that actually matters for saving the planet.

Technical Summary: Inducing Permutation Invariant Priors in Bayesian Optimization for Carbon Capture and Storage Applications

Problem Statement

The paper addresses the challenge of optimizing the design and operation of Carbon Capture and Storage (CCS) projects, specifically the strategic placement of injection and production wells. This optimization involves navigating a high-dimensional, expensive black-box objective function where each evaluation requires a computationally intensive, high-fidelity reservoir simulation.

A critical structural characteristic of this problem is permutation invariance. Under "group control" strategies, the simulator output depends on the set of injector well coordinates and the set of producer well coordinates, rather than the specific ordering of these coordinates within their respective groups. Standard Bayesian Optimization (BO) surrogates, such as Gaussian Processes (GPs) with standard kernels, treat inputs as ordered vectors. Consequently, they fail to recognize that different permutations of the same physical well layout represent the same design. This lack of symmetry encoding leads to:

Redundant learning: The model treats equivalent physical designs as distinct inputs.
Distorted posterior geometry: The surrogate fails to collapse the design space correctly.
Reduced sample efficiency: The optimization process requires more expensive simulations to converge, which is prohibitive in CCS applications.

Methodology

The authors propose a novel approach to enforce permutation invariance directly within the surrogate model, alongside a learned-invariant baseline.

1. GP-Perm: A Divergence-Based Permutation-Invariant Kernel

The primary contribution is GP-Perm, a Gaussian Process surrogate that encodes permutation invariance at the kernel level.

Input Structure: The design variable $x$ is structured as $(v_x, I_x, P_x)$ , where $v_x$ represents auxiliary continuous controls (e.g., group target rates), and $I_x$ and $P_x$ are unordered sets of injector and producer well coordinates, respectively.
Composite Distance Metric: The kernel is built upon a composite squared distance $D^2(x, x')$ $D^{2} (x, x^{'})$ that aggregates:
- Standard Euclidean distance for the auxiliary vector $v_x$ .
- Sinkhorn Divergence ( $S_\epsilon$ ): A stable, entropic optimal transport divergence applied to the injector set $I_x$ , the producer set $P_x$ , and an explicitly constructed interaction set $R_x$ .
- Interaction Set ( $R_x$ ): Defined as the set of relative vectors between all injector-producer pairs ( $p_b - i_a$ ). This explicitly captures cross-group geometric relationships (spacing and alignment) which are crucial for CCS performance but are lost if sets are treated independently.
Kernel Construction: The final kernel is a Matérn kernel applied to this composite distance: $k(x, x') = \sigma^2 \mathcal{M}_\nu(D(x, x'))$ .
Properties: Because the Sinkhorn divergence depends only on the induced empirical measures of the sets, the kernel is invariant to any permutation of elements within $I_x$ or $P_x$ . The authors implement numerical safeguards (staged jitter, eigenvalue clamping) to ensure stability during Cholesky factorization.

2. DKL-DS: A Learned Invariant Baseline

To benchmark the explicit kernel approach, the authors implement DKL-DS (Deep Kernel Learning with Deep Sets).

Architecture: A Deep Sets encoder processes the input sets ( $I_x, P_x, R_x$ ) using a shared element-wise map followed by invariant pooling (concatenation of mean, max, std, and power means). The auxiliary vector is processed via an MLP.
Latent Space: The fused embeddings form a latent representation $z$ , upon which a standard GP is placed.
Training: The model is trained end-to-end by maximizing the marginal log-likelihood, allowing gradients to flow through both the neural network and GP hyperparameters.

3. Evaluation Protocol

The methodology is evaluated across:

Synthetic Benchmarks: Seven permutation-invariant objectives (e.g., Max Area Coverage, Distribution Matching) to stress-test set reasoning capabilities.
CCS-like Synthetic Benchmark: A custom two-set objective mimicking injector-producer geometric coupling.
Realistic Case Study: Optimization of well placement in the Johansen formation using the OPM-Flow reservoir simulator. The objective is to maximize a regularized Net Present Value (NPV).

Key Results

Synthetic Benchmarks

Performance: No single surrogate dominated all benchmarks, highlighting the task-dependency of inductive biases. However, GP-Perm achieved the best performance (in terms of Area Under the Curve and final best value) on the "Facility Location" and "Soft k-Medoids" benchmarks.
Comparison: Permutation-invariant surrogates (GP-Perm and DKL-DS) generally outperformed non-invariant baselines (standard GP and DKL) under identical budgets, confirming the benefit of symmetry encoding.

CCS Case Study (Johansen Formation)

Sample Efficiency: GP-Perm consistently outperformed all baselines, including non-invariant GPs, non-invariant DKL, and engineered set-kernel baselines (Double-Sum and Deep-Embedding kernels).
Metrics: GP-Perm achieved the highest mean Area Under the Curve (7.717) and the best final objective value (0.387) in the Johansen study, despite a tight evaluation budget (10 iterations, batch size 2).
Posterior Diagnostics: Analysis of posterior uncertainty and acquisition behavior revealed that DKL-based models (including DKL-DS) exhibited signs of overconfidence (very small posterior standard deviations) and large standardized residuals, suggesting representation drift in the small-data regime. In contrast, GP-Perm maintained a more moderate uncertainty profile and better-calibrated residuals, indicating a robust balance between geometric discrimination and uncertainty contraction.

Significance and Claims

The paper claims that explicitly encoding permutation invariance is critical for efficient Bayesian Optimization in CCS well design. The authors argue that:

Symmetry Exploitation: Standard surrogates waste capacity modeling physically meaningless distinctions between permutations. By enforcing invariance, the surrogate collapses the design space, leading to smoother posteriors and more reliable acquisition decisions under tight budgets.
Superiority of Explicit Kernels: While learned embeddings (DKL-DS) are flexible, they can suffer from instability and representation drift when retrained repeatedly on small, non-i.i.d. BO datasets. The explicit GP-Perm kernel, built on stable divergences, provides a more reliable default for small-data regimes where geometric sensitivity is paramount.
Importance of Interaction Sets: The inclusion of the interaction set ( $R_x$ ) in both GP-Perm and DKL-DS is highlighted as a crucial design choice. It allows the model to capture cross-group geometric relationships (injector-producer spacing) that are essential for operational outcomes like pressure management and CO2 recycling risks.

The study concludes that for problems with exact exchangeability (like group-controlled well placement), inducing permutation-invariant priors is typically beneficial. However, the authors note that strict invariance should be relaxed if wells have specific, non-exchangeable constraints. The work provides a practical framework for handling unordered sets in high-fidelity physical simulations, offering a path toward more efficient decision-making in complex engineering systems.