🔬 materials science

Measuring trainable degrees of freedom in materials graph neural networks: a random-subspace intrinsic dimension analysis

This paper introduces trainable-degree dependence as a new characterization for materials graph neural networks by using random-subspace intrinsic dimension analysis to reveal how different architectures and prediction tasks vary in their sensitivity to the number of trainable degrees of freedom required to achieve high performance, distinguishing these demands from final accuracy alone.

Original authors: Shehroz Ahmad Shoaib, Kangming Li

Published 2026-09-30

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Shehroz Ahmad Shoaib, Kangming Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the quest to design better materials, scientists have turned to a powerful new tool: artificial intelligence that can "see" the invisible architecture of matter. These systems, known as graph neural networks, treat a material not as a solid block, but as a map of atoms connected by bonds, much like a subway system where stations are atoms and tracks are the chemical links between them. By studying these maps, the computer learns to predict how a material will behave—whether it will conduct electricity, how hard it is to crush, or how it vibrates. For years, the only way to judge if one of these AI models was good was to look at its final score: how close its prediction was to the real, measured value in a lab. If the error was small, the model was considered a success. But this single number hides a crucial mystery. It does not tell us how the model actually learned the answer, or how much of its own internal "brainpower" it needed to get there.

A team of researchers at King Abdullah University of Science and Technology decided to look behind the curtain. They wanted to understand not just the final result, but the path the model took to get there. Imagine a student taking a test. If two students both get an A, we assume they are equally smart. But what if one student needed to memorize every single fact in the textbook to get that A, while the other only needed to understand a few core concepts? The first student is fragile; if you remove a few pages from their textbook, they fail. The second student is robust; they can still pass even with a much smaller book. The researchers realized that current AI models for materials are often treated like the first student. We know they get the right answer, but we don't know if they are relying on a vast, complex web of connections or if they could solve the problem with a much simpler mind. To find out, they developed a new way to test these models by deliberately limiting their ability to learn.

The team took three popular AI models used for predicting material properties and forced them to learn using only a tiny fraction of their available "thinking space." Normally, these models have hundreds of thousands of adjustable knobs that they can turn to improve their predictions. The researchers locked most of these knobs in place and allowed the models to only adjust a small, random selection of them. They started with a very small selection—just a tiny sliver of the total available space—and watched how well the models performed. Then, they slowly unlocked more knobs, giving the models more freedom to learn, until they had access to their full capacity. By measuring how the models' performance improved as they regained these freedoms, the researchers could draw a "recovery curve" for each task. This curve revealed a hidden truth: some material properties are easy to learn and require very little mental effort, while others are incredibly difficult and demand the full power of the model's entire brain.

The results showed that different materials behave in surprisingly different ways. Predicting whether a metal is magnetic or calculating its bulk stiffness turned out to be relatively simple. These models could recover nearly perfect accuracy even when they were only allowed to use about five to ten percent of their total adjustable parameters. It was as if these tasks could be solved with a small, focused set of rules. However, predicting the energy required to form a material or the size of its electronic band gap was much harder. These tasks showed a strong dependence on the specific design of the AI model. One model, called ALIGNN, could solve the formation energy problem using only fifteen percent of its capacity, while the other two models needed to unlock half of their parameters to reach the same level of accuracy. This suggested that the way a model is built matters deeply for certain tasks, and that a model's final score alone does not tell the whole story of its efficiency.

The most striking finding came from the study of how materials vibrate, known as phonon prediction. This task was the most sensitive to the restriction of learning freedom. As the researchers locked down more of the model's parameters, the performance of the models generally degraded significantly. However, the behavior varied by architecture: one model, DimeNet++, was less accurate in absolute terms than the best-performing model, ALIGNN, yet it showed a less pronounced average drop in performance over part of the dimensional sweep. This contrast highlighted a trade-off: the model with the lowest final error was not necessarily the one that degraded the least when its learning freedom was restricted. This indicated that predicting vibrations requires a highly coordinated effort across the entire network of the model's parameters. It is not a task that can be solved by a few clever tricks; it demands the full, unobstructed access to the model's entire optimization space. Furthermore, the researchers found that this sensitivity varied by the amount of data the model was trained on. For predicting band gaps, adding more data actually made the task harder for the model to solve with limited resources, requiring a larger fraction of the model's parameters to achieve the same level of accuracy. This suggests that as the data grows more complex, the model needs to unlock more of its internal machinery to handle the new information.

The study also challenged a common assumption about how these models scale up. When the researchers made the models larger by increasing their size, they found that the absolute number of parameters needed to solve a problem grew in direct proportion to the model's size. A model that was twelve times larger needed roughly twelve times as many adjustable knobs to reach its full potential. This means that simply making a model bigger does not automatically make it more efficient; it just gives it a larger pool of resources to draw from. The fraction of the model needed to solve the problem stayed roughly the same, but the total amount of "brainpower" required increased. This finding suggests that the complexity of the task is tied to the specific way the model is built, rather than being a fixed property of the material itself.

Ultimately, this work provides a new lens for understanding artificial intelligence in science. It shows that a high score on a test is not the only measure of a model's quality. A model that reaches a high score by using almost all of its available resources is fundamentally different from one that reaches the same score with a tiny fraction of its capacity. The former is fragile and may struggle when faced with new or slightly different data, while the latter is robust and efficient. By mapping out how much freedom a model needs to learn different material properties, scientists can now choose the right tool for the job, design better datasets, and understand the true complexity of the physical world they are trying to simulate. The final score tells us what the model can do, but this new method tells us how it does it, revealing the hidden architecture of learning itself.

Technical Summary: Measuring Trainable Degrees of Freedom in Materials Graph Neural Networks

Problem Statement
In materials property prediction, the evaluation of Graph Neural Networks (GNNs) is currently dominated by final predictive accuracy metrics, such as Mean Absolute Error (MAE) or ROC-AUC. While indispensable, these aggregate scores fail to reveal how strongly a model's performance depends on access to its full trainable parameter space. Specifically, standard benchmarks do not distinguish whether a low error rate arises from a favorable architectural inductive bias, dominant chemical trends in the data, or the necessity of utilizing a vast number of independent optimization directions. Furthermore, existing model-scaling studies and pruning methods do not adequately characterize how performance recovers as independent trainable degrees of freedom are progressively restored. This limitation hinders the understanding of dataset design (e.g., whether larger datasets introduce new constraints or merely increase sampling density) and model design (e.g., whether architectural choices trade off absolute accuracy against dimensional demand).

Methodology
The authors introduce trainable-degree dependence as a complementary characterization of materials GNN learning, utilizing random-subspace intrinsic dimension analysis. The core methodology involves:

Random-Subspace Restriction: Instead of optimizing all $D$ parameters of a model, training is restricted to a randomly oriented affine subspace of dimension $d \le D$ . The model parameters $\theta$ are defined as $\theta = \theta_0 + A\phi$ , where $\theta_0$ is a fixed initialization, $A$ is a fixed random projection matrix, and $\phi$ represents the $d$ trainable coordinates.
Dimensional Sweeps: For each model-task combination, the dimension $d$ is swept from small fractions of $D$ up to the full dimension. This generates a "dimensional-response curve" plotting normalized predictive performance against the fractional subspace dimension ( $d/D$ ).
Threshold Definition: A scalar summary, the threshold dimension $d_{10}$ , is defined as the dimension required to recover performance within a 10% relative tolerance of the full-subspace ( $d=D$ ) reference.
Experimental Setup: The analysis was applied to three atomistic GNN architectures—CGCNN, ALIGNN, and DimeNet++—across six tasks: formation energy, band gap, metallicity classification, log bulk modulus, refractive index, and phonon frequency. Experiments included:
- Cross-architecture comparisons using compact, parameter-matched models.
- Dataset-size sweeps (full, 1/4, and 1/10 subsets) for formation energy, band gap, and log bulk modulus.
- Model-width sweeps (varying channel width) for ALIGNN and DimeNet++ to test scaling behavior.
- Use of Fastfood structured projections to ensure computational tractability.

Key Contributions and Results
The study reveals that trainable-degree demand is a distinct, model-relative property that is not determined by final accuracy alone. Key findings include:

Distinct Dimensional Signatures: Different tasks exhibit qualitatively different recovery curves.
- Metallicity classification and log bulk modulus regression recover near-reference performance from small fractional subspaces (~5–10% of $D$ ) with minimal separation between architectures.
- Formation energy and band gap prediction show strong architecture dependence. For instance, ALIGNN recovers band-gap performance with ~10% of the parameter space, whereas CGCNN and DimeNet++ require ~50%.
- Phonon prediction is the most sensitive to dimensional restriction, showing sustained degradation and high run-to-run variability, particularly for smaller datasets.
Dataset Size Effects: The impact of increasing dataset size on dimensional demand is property-dependent.
- For band gap prediction, larger datasets require larger fractional subspaces to recover their own improved full-subspace performance (e.g., DimeNet++ requires ~40–50% for the full dataset vs. ~5% for 1/10 subsets). This suggests larger datasets impose additional constraints on the learned mapping.
- For formation energy and log bulk modulus, the fractional thresholds remain relatively stable across dataset sizes, with larger datasets primarily improving absolute accuracy and reproducibility rather than shifting the mean dimensional demand.
Model Scaling Behavior: In width-sweep experiments for log bulk modulus prediction:
- The fractional threshold ( $d_{10}/D$ ) remains approximately stable as model size increases (e.g., ALIGNN requires ~10% of parameters regardless of whether $D$ is 86k or 1.06M).
- However, the absolute threshold ( $d_{10}$ ) scales linearly with model size. This contrasts with previous findings in fully connected networks where absolute intrinsic dimension was invariant to model size. The authors conclude that for these atomistic GNNs, additional parameters do not merely create redundant solution directions; the absolute number of required trainable degrees of freedom grows with model size.
Accuracy-Dimensional Demand Trade-off: The analysis uncovers trade-offs invisible to final MAE. For example, in phonon prediction, the architecture with the lowest full-subspace error (ALIGNN) is not necessarily the one with the weakest relative degradation under dimensional restriction.

Significance and Claims
The paper positions random-subspace intrinsic dimension analysis as a targeted stress test and a complementary diagnostic tool for materials GNNs. Its significance lies in:

Decoupling Accuracy from Complexity: It separates the attainable endpoint accuracy from the "trainable-dimensional demand" required to reach it.
Revealing Hidden Dependencies: It exposes how specific model-task pairs interact with the optimization space, distinguishing between models that are robust to parameter restriction and those that rely heavily on the full parameter space.
Guiding Dataset and Model Design: It provides a framework to test hypotheses about dataset redundancy and diversity. For instance, the observation that band-gap models require larger fractional subspaces as data grows suggests that larger datasets may introduce non-redundant structural constraints.
Refining Intrinsic Dimension Interpretation: The results clarify that "dimensional demand" is a property of the model, dataset, task, and training protocol combined, rather than an intrinsic property of the material property itself.

The authors modestly note that the method does not identify a compressed model or assign an architecture-independent physical complexity to a property. Instead, it reveals how a specific configuration interacts with its available optimization space. The study is limited to compact, one-layer model configurations for computational feasibility, and the scalar threshold $d_{10}$ may be less robust for non-monotonic responses or coarse dimension grids. Nevertheless, the analysis offers a new axis for understanding materials learning systems by quantifying the stability and dependence of performance on trainable degrees of freedom.

Technical Summary: Measuring Trainable Degrees of Freedom in Materials Graph Neural Networks

More like this