📊 statistics

Space-Filling One-Factor-At-A-Time Designs

This paper proposes a new class of screening designs that successfully combines the factor identification capabilities of MOFAT designs with the space-filling properties necessary for effective surrogate modeling in deterministic computer experiments.

Original authors: Wei-Yang Yu, V. Roshan Joseph

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wei-Yang Yu, V. Roshan Joseph

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a giant, dark warehouse filled with thousands of switches. Your goal is to figure out which few switches actually turn on the lights (the "important factors") and which ones are just dead weight. Once you know which switches matter, you want to build a perfect map of how the lights behave so you can predict exactly what will happen if you flip any combination of them.

This paper is about finding the best way to flip those switches to get the job done quickly and accurately.

The Problem: Two Bad Options

The authors explain that scientists usually face a dilemma with two existing strategies, both of which have flaws:

The "Spray and Pray" Approach (Space-Filling Designs):
Imagine throwing darts at the warehouse walls completely at random, trying to hit every single inch of the floor equally. This is great for making a detailed map of the whole room. However, if you only have a limited number of darts, you might miss the specific switches that actually turn on the lights. You waste time checking dead switches, and you might not have enough darts left to figure out exactly how the important ones work.
The "One-at-a-Time" Approach (OFAT Designs):
This is the classic method: flip Switch A, see what happens. Then flip Switch B, see what happens. You do this one by one. This is excellent for finding the "important" switches quickly. However, the problem is that you usually only check the switches near the edges of the room or in a very narrow path. You end up with a map that has huge, dark, unexplored holes in the middle. If you try to predict what happens in the middle of the room, your map fails because you never looked there.

The New Solution: The "SOFT" Design

The authors propose a new method called SOFT (Space-filling One-Factor-at-a-Time).

Think of SOFT as a smart, systematic tour guide.

Like the "One-at-a-Time" detective, it flips switches one by one to quickly identify which ones are important.
But unlike the old method, it doesn't just wander randomly or stick to the edges. It carefully plans its path so that it covers the entire warehouse floor evenly, leaving no dark corners unexplored.

It's like a robot vacuum that is programmed to not only find the dirtiest spots (the important factors) but also to ensure it cleans every single square inch of the floor without missing a spot or bumping into the same wall twice.

How They Built It

The authors didn't just guess; they used math to build the perfect path:

The Levels: They decided exactly how far to move each switch (not too little, not too much) to ensure they cover the whole range of possibilities.
The Structure: They figured out the best order to flip the switches. They tested two main patterns:
- Standard: Always starting from the same "home base" position.
- Strict: Moving from the last position of the previous switch to the next one (like a continuous chain).
The Optimization: They used a computer algorithm to shuffle the starting positions until they found the arrangement that covered the most ground with the least amount of wasted movement.

What They Found

The authors tested their new SOFT design against the old methods using computer simulations (mathematical "test drives").

Better Maps: When they tried to build a prediction map (a "surrogate model") of how the system works, the SOFT design created the most accurate maps.
Handling "Dead" Switches: In real life, many systems have "effect sparsity," meaning only a few factors actually matter. The old "Spray and Pray" methods struggled when there were many useless factors, getting confused and making poor predictions. The SOFT design handled this perfectly, ignoring the useless factors while still mapping the whole room.
Speed: Creating these designs was much faster than creating the complex "Spray and Pray" designs.

The Bottom Line

The paper claims that the SOFT design is the best of both worlds. It keeps the superpower of quickly finding the important factors (screening) while also ensuring you have a complete, gap-free map of the entire system (space-filling).

In the authors' words, this is the first time such a design has been created specifically for computer experiments where the goal is to understand complex, non-linear interactions. It allows scientists to get a better understanding of their systems with fewer experiments and more reliable predictions.

Technical Summary: Space-Filling One-Factor-At-A-Time Designs

Problem Statement
In deterministic computer experiments, space-filling designs (e.g., Latin Hypercube Designs) are standard for exploring input spaces to build robust surrogate models, such as Gaussian Processes (GPs). However, these designs are often inefficient for factor screening because they vary all factors simultaneously, making it difficult to isolate causal effects when only a small subset of inputs is influential (the "effect sparsity" assumption). Conversely, One-Factor-At-a-Time (OFAT) designs are effective for screening but historically suffer from poor space-filling properties, leading to gaps in the input space that degrade the accuracy of surrogate modeling. Existing screening designs, such as Morris designs and the recently proposed Maximum One-Factor-At-a-Time (MOFAT) designs, improve screening capability but fail to provide adequate coverage of the interior of the input space, often clustering points near boundaries or leaving large unexplored regions.

Methodology
The authors propose a new class of designs called Space-filling One-Factor-at-a-Time (SOFT) designs. These designs aim to optimize space-filling properties while retaining the screening structure of OFAT designs. The methodology involves three key components:

Optimization Criterion: Instead of relying on simple distance-based metrics (like minimax distance) which may not account for extrapolation errors in GP models, the authors optimize a criterion based on the Maximum Mean Squared Error (MMSE) of a GP surrogate. Specifically, they maximize the quadratic form $Q(D; \alpha, \theta) = \min_{x \in E} \{r(x)'R^{-1}r(x)\}$ , where $E$ is an evaluation set comprising uniform Monte Carlo samples and representative vertices of the input hypercube. This ensures the design minimizes the worst-case prediction error under a Multiplicative Inverse Multiquadric (MIM) kernel.
Factor Levels and Pairing: To maintain screening capability, the design uses $2l$ equally spaced levels for each factor. The authors derive an optimal pairing strategy (Proposition 1) where levels are paired to maximize the total perturbation magnitude, ensuring accurate estimation of total Sobol' indices. They identify a robust starting offset ( $a = 0.125/l$ ) for the levels that performs well across varying correlation structures.
Structural Optimization: The paper addresses two OFAT structures:
- Standard Structure: All OFAT trajectories originate from a common base run matrix $A^{(0)}$ .
- Strict Structure: Trajectories are chained sequentially.
  For the strict structure, where evaluating all permutations is computationally infeasible, a greedy algorithm is proposed to construct the sequence of factor changes that maximizes the space-filling criterion. For both structures, a deterministic local search algorithm is used to optimize the base runs ( $A^{(0)}$ ) by swapping rows to improve the $Q$ criterion.

Key Contributions

Novel Design Class: Introduction of SOFT designs, which are the first to explicitly integrate strong space-filling properties with the screening capabilities of OFAT designs in the context of computer experiments.
Optimization Framework: Development of a specific optimization framework that balances the need for screening (via OFAT structure and perturbation magnitude) with the need for global prediction accuracy (via MMSE-based space-filling criteria).
Algorithmic Efficiency: Presentation of efficient algorithms (greedy construction and local search) to generate these designs, noting that while they require optimization, they are computationally more efficient than generating high-dimensional space-filling designs like MaxPro.

Results
Numerical studies comparing SOFT designs against MmLHD, MaxPro, and MOFAT designs across dimensions $p=2$ to $20$ yield the following findings:

Space-Filling Properties: SOFT designs achieve space-filling performance (measured by criteria $Q$ and $\Phi$ ) comparable to classical space-filling designs (MmLHD and MaxPro) and significantly superior to MOFAT designs, particularly in high-dimensional settings. The strict structure generally offers better space-fillingness than the standard structure as dimensions increase.
Surrogate Modeling Accuracy: In terms of out-of-sample Mean Squared Error (MSE) for benchmark functions (g-function, Levy, Ackley):
- SOFT designs consistently achieve the lowest MSE across most scenarios, including those with effect sparsity (inert factors).
- While pure space-filling designs (MmLHD, MaxPro) struggle when many factors are inert, and MOFAT designs struggle when factors are active or in high dimensions, SOFT designs remain robust in both regimes.
- The standard structure of SOFT designs often provides the most stable predictive performance.
Computational Cost: SOFT designs are significantly faster to generate than MmLHD and MaxPro designs, though slightly more costly than MOFAT designs due to the optimization step.

Significance and Claims
The paper claims that SOFT designs offer a "clear advantage" over existing methods by resolving the trade-off between screening and space-filling. The authors assert that SOFT designs are more robust for surrogate modeling than traditional space-filling designs, a result they describe as "surprising" given the conventional wisdom that space-filling is paramount for computer experiments. They position SOFT as the first design of its kind in computer experiments that successfully integrates screening and response surface modeling capabilities, potentially impacting how complex systems are explored and modeled when factor sparsity is present. The work is supported by the U.S. National Science Foundation.