Diversity-Aware Batch-Mode Active Learning for Efficient Sampling in Data-Driven Constitutive Modeling

This paper proposes a diversity-aware batch-mode active learning strategy that utilizes a committee of support vector classifiers and a cosine-similarity metric to efficiently generate non-redundant, informative datasets for constitutive modeling, thereby achieving predictive accuracy comparable to sequential methods while significantly reducing the number of machine learning retraining cycles required.

The Gist

The Big Picture: Mapping a Hidden Shape

Imagine you are trying to draw a map of a mysterious, invisible island. You know the island exists, but you can't see it. You only know that if you step on certain spots, you sink into the water (plastic deformation), and if you step on others, you stay dry on land (elastic behavior). The line where the water meets the land is called the yield surface.

In the world of materials science, this "island" exists in a complex, six-dimensional space (which is impossible for humans to visualize). To learn what this island looks like, scientists usually have to send out "scouts" to test specific points. However, sending out scouts one by one is slow, and sending them out randomly is wasteful—you might test the same flat beach ten times while missing the jagged cliffs.

This paper introduces a smarter way to send out these scouts.

The Problem: The "Retraining" Bottleneck

The researchers use a computer program (a machine learning model) to guess the shape of the island.

1. The Old Way (Sequential): The computer picks one spot, sends a scout, gets the answer, updates its map, picks the next spot, updates the map again, and so on.
   • The Analogy: Imagine a teacher who stops the class every time a student asks a question to rewrite the entire lesson plan. It's accurate, but it takes forever because the teacher is constantly stopping to rewrite.
2. The Issue: In this specific field, "updating the map" (retraining the computer model) is very expensive and time-consuming. If you have to do it 200 times, the project drags on.

The Solution: The "Diversity-Aware" Squad

The authors propose a new strategy called Batch-Mode Active Learning. Instead of picking one scout at a time, they pick a whole team (a "batch") of scouts to send out at once.

However, there is a trap: If you just pick the 5 most confusing spots, your team might all end up standing in the same small puddle, giving you the same answer five times. This is called redundancy.

To fix this, the authors created a "Diversity-Aware" system. Think of it as a team captain with two rules for picking the squad:

1. Rule 1 (Uncertainty): "Pick the spots where our current map is most confused." (This is the "Query-by-Committee" part: imagine a group of experts arguing about where the island is; if they disagree, that's a good place to look).
2. Rule 2 (Diversity): "Make sure the scouts in this team are spread out." (This is the "Cosine Similarity" part: if Scout A is going North, don't send Scout B to go North-North-East. Send them East or South instead).

How It Works in Practice

The researchers tested this on a simulated material (using a mathematical formula called the Hill criterion as a "truth-teller").

• The Setup: They started with a small, random map.
• The Process:
  • They asked the computer to pick a batch of 2, 3, or 4 new directions to test.
  • The computer ensured these directions were far apart from each other (diverse) but still in areas where the computer was unsure (informative).
  • They sent all these scouts out at the same time.
  • Once the answers came back, they updated the map once for the whole batch.

The Results: Faster Maps, Same Accuracy

The paper found three main things:

1. No Loss in Quality: Sending a team of scouts didn't make the map worse. The final result was just as accurate as sending scouts one by one.
2. Huge Time Savings: Because they only had to "rewrite the lesson plan" (retrain the model) once for every 2, 3, or 4 scouts, the process was much faster.
   • The Analogy: If the teacher has to rewrite the lesson plan 100 times for 100 students, it takes a long time. But if the teacher rewrites it 25 times for groups of 4 students, the class finishes in a quarter of the time, and the students learn just as well.
3. No Clumping: The "Diversity" rule worked perfectly. The scouts didn't crowd into the same spot; they explored the whole island evenly.

Why This Matters

In the real world, getting "ground truth" data (the answers from the scouts) often requires running expensive, high-tech computer simulations that take hours or days.

• Sequential: Run 1 simulation -> Wait -> Update Model -> Run 1 simulation -> Wait... (Very slow).
• Batch Mode: Run 4 simulations at the same time (on different computers) -> Wait -> Update Model once.

By using this "Diversity-Aware" batch strategy, scientists can build accurate models of how materials behave much faster, without wasting time testing the same things over and over again. The paper concludes that this is a highly efficient way to sample complex stress spaces, specifically reducing the time it takes to solve these problems.

Technical Summary

Technical Summary: Diversity-Aware Batch-Mode Active Learning for Constitutive Modeling

Problem Statement

In data-driven constitutive modeling, particularly for elastoplastic materials, the goal is to learn the yield function—a manifold separating elastic and plastic regimes in a high-dimensional stress space (typically six-dimensional). Traditional static sampling strategies (e.g., uniform sampling or fixed loading directions) often suffer from inefficiency in high-dimensional spaces, leading to redundant evaluations in well-resolved regions and insufficient coverage in complex areas.

While active learning (AL) addresses this by adaptively selecting informative data points, standard AL approaches are typically sequential: a single point is queried, and the model is retrained immediately. This sequential nature incurs substantial computational overhead when model retraining is expensive. Although batch-mode AL (selecting multiple points per iteration) exists in broader machine learning, its application to constitutive modeling is scarce. Existing batch methods often lack mechanisms to ensure diversity within a selected batch, leading to clustering of queries in specific regions and redundant information gain.

Methodology

The authors propose a diversity-aware batch-mode Query-by-Committee (QBC) active learning strategy designed to generate maximum information content at minimum cost. The methodology integrates the following components:

1. Surrogate Model (ML Yield Function):

   • The yield surface is approximated using a Support Vector Classifier (SVC) with a Radial Basis Function (RBF) kernel.
   • The problem is cast as a binary classification task: classifying stress states as elastic ($f(\sigma) < 0$) or plastic ($f(\sigma) \geq 0$).
   • Ground-truth labels are generated using Hill's anisotropic yield criterion as a reference oracle. For a given loading direction, the oracle determines the yield onset, and points are labeled based on radial scaling relative to this onset.

2. Committee-Based Uncertainty (QBC):

   • A committee of $N$ SVC models is trained on the current dataset.
   • Diversity within the committee is induced by training each member on a different random 80% split of the data.
   • Uncertainty is quantified by the variance of predictions across the committee at a fixed probe stress level along a candidate loading direction. High variance indicates regions where the model is uncertain (near the yield surface).

3. Diversity-Aware Batch Selection:

   • To select a batch of $b$ directions per iteration, the authors introduce a two-step selection process that balances uncertainty and diversity:
     • First Direction: Selected by maximizing the committee variance (standard QBC).
     • Subsequent Directions ($i = 2 \dots b$): Selected by minimizing a combined objective function: $\text{Var}(\hat{\sigma}) \times D_i(\hat{\sigma})$.
   • The Diversity Term ($D_i$) is based on cosine similarity. It penalizes candidate directions that are angularly similar to directions already selected in the current batch. Specifically, $D_i(\hat{\sigma}) = -1 + \sum_{j=1}^{i-1} (\hat{\sigma} \cdot \hat{\sigma}_j^*)$.
   • This mechanism ensures that while the batch targets high-uncertainty regions, the selected points within that batch are geometrically distinct, preventing redundancy.

Key Contributions

• Novel Selection Criterion: The paper introduces a cosine-similarity-based metric that complements the uncertainty criterion in QBC. This allows for the selection of multiple informative, non-redundant queries per iteration.
• Efficient Batch-Mode Implementation: The strategy enables concurrent generation of informative datasets and reduces the number of machine-learning retraining cycles, which is critical when retraining is computationally expensive.
• Benchmarking in Constitutive Modeling: The method is rigorously benchmarked for stress-space sampling in data-driven constitutive modeling, demonstrating robustness across different batch sizes ($b=2, 3, 4$).

Results

The proposed method was evaluated against a sequential variance-only baseline using Matthew's Correlation Coefficient (MCC) on a held-out test set.

• Within-Batch Diversity: The strategy successfully maintains high intra-batch diversity. For batch size $b=2$, the mean cosine distance between selected directions remained significantly higher than random pairs (mean $\approx 1.62$). Similar diversity was maintained for $b=3$ and $b=4$, though geometric constraints naturally reduced the marginal diversity of later selections in the batch.
• Uncertainty Reduction: The method rapidly reduces committee variance (uncertainty) in the early iterations, stabilizing near zero as the yield surface is learned. This reduction occurs without sacrificing directional exploration.
• Query Efficiency vs. Update Efficiency:
  • Query Efficiency: Batch-mode sampling preserves the sample efficiency of sequential AL. For a fixed number of oracle queries, batch-mode and sequential methods achieve comparable MCC values.
  • Update Efficiency: Batch-mode sampling significantly outperforms sequential AL when measured by the number of retraining cycles (iterations). Larger batches ($b=3, 4$) achieve higher MCC for the same number of retraining cycles, effectively doubling or tripling the information gained per expensive model update.
• Redundancy Analysis: Global redundancy checks (Appendix A) confirm that the selected directions do not collapse into duplicate queries, even for larger batch sizes. The fraction of near-duplicate pairs (cosine similarity $\geq 0.90$) remains low ($< 2.7\%$).

Significance and Claims

The paper claims that the proposed diversity-aware batch-mode QBC strategy is an efficient strategy for stress-space sampling in data-driven constitutive modeling. Its primary significance lies in:

1. Reducing Time-to-Solution: By reducing the number of costly retraining cycles, the method significantly lowers wall-clock time, particularly in settings where model retraining dominates the computational cost.
2. Enabling Parallelism: In simulation-driven settings where ground-truth evaluations (e.g., high-fidelity simulations) are expensive and can be parallelized, the method allows for concurrent data collection within each iteration, offering potential for even greater time savings.
3. Robustness: The approach handles different batch sizes robustly, maintaining high predictive accuracy comparable to sequential active learning while avoiding the redundancy pitfalls of naive batch selection.

The authors note that while the benchmark used an inexpensive analytical oracle (Hill's criterion), the method is designed for scenarios where ground-truth generation is costly. In such practical applications, the reduction in retraining cycles and the ability to parallelize oracle queries represent the primary efficiency gains. The study suggests $b=4$ as a practical upper bound, as larger batches increase the risk of redundancy and may delay model bias correction.