Factorial clinical trials in the presence and absence of plausible statistical interactions between treatment factors. A historical review of the methodological literature

Problem
Factorial clinical trials, which randomize participants to combinations of two or more treatment factors, are subject to confusion and misunderstanding regarding their appropriate use and analysis. Two distinct "schools of thought" exist in the literature, creating conflicting guidance. The first school, originating in the UK and dominated by the work of Peto (1978) and subsequent clinical trialists, advocates for k-in-1 factorial trials. In this approach, factorial designs are used to evaluate multiple whole interventions (e.g., drugs) simultaneously with the sample size of a single parallel-group trial, but only under the strict assumption that no statistical interactions exist between the treatment factors. The objective is to estimate the "separate effects" of each intervention when administered in isolation.

The second school, originating from the Design of Experiments (DoE) field (Fisher, Yates) and recently revitalized by the Multiphase Optimisation Strategy (MOST) framework, recommends factorial trials for both the presence and absence of interactions. In this view, factorial experiments are designed to robustly estimate "main effects" (the average effect of a component across all levels of other components) and interactions. The objective is often to screen, optimize, or evaluate complex interventions where components may interact.

The core problem is that the prevailing guidance in clinical trials often conflates these distinct objectives, leading to inappropriate sample size calculations, analysis strategies, and interpretations. Specifically, the assumption of no interaction is often treated as a universal requirement for factorial trials, whereas it is only a requirement for the specific efficiency gains sought in k-in-1 trials. This conflation obscures the fact that factorial designs are efficient for estimating main effects and interactions even when interactions are present, provided the trial is designed and analyzed with those specific estimands in mind.

Methodology
This paper provides a historical review and methodological synthesis of the literature on factorial clinical trials. The authors:

1. Historically traced the origins of both schools of thought, from Fisher and Yates in the 1930s (DoE) to Peto and subsequent clinical trialists in the 1970s and 1980s.
2. Defined and contrasted the objectives, estimands, treatment contrasts, and decision-making approaches for k-in-1 factorial trials versus other factorial trials (screening, optimization, and evaluation).
3. Mathematically derived the properties of estimators for both schools. The authors compare the properties of estimators under two parameterizations:
   • Effect coding (-1, +1): Used in DoE and MOST to estimate main effects and interactions. They demonstrate that under a balanced design, main effects and interactions are orthogonal (uncorrelated) and estimated with equal precision, regardless of whether interactions are included in the model.
   • Dummy coding (0, 1): Often used in clinical trials to estimate "separate effects." They show that in k-in-1 trials, estimators for separate effects are correlated, and their precision changes significantly when interactions are added to the model. If interactions are present, the precision of separate effects in a dummy-coded maximal model is lower than in an additive model.
4. Illustrated the impact of these different approaches using an empirical example (a simplified version of Kingkaew et al., 2020) involving text messages for smoking cessation. They fitted additive and maximal models using both dummy and effect coding to demonstrate how the choice of estimand and coding scheme alters the interpretation of coefficients, standard errors, and statistical significance.

Key Contributions

• Clarification of Schools of Thought: The paper explicitly delineates the two schools of thought, arguing that the "no interaction" requirement applies only to k-in-1 factorial trials designed to estimate separate effects efficiently. It asserts that this requirement does not apply to factorial trials designed to estimate main effects and interactions (screening, optimization, evaluation).
• Estimands and Contrasts: The authors clarify that the choice of estimand dictates the analysis.
  • For k-in-1 trials, the estimands are separate effects (e.g., effect of A in the absence of B).
  • For DoE/MOST trials, the estimands are main effects (average effect of a component) and interactions.
• Properties of Estimators: The paper provides a rigorous mathematical comparison showing that:
  • In effect-coded models (DoE/MOST), main effects and interactions are orthogonal and maintain equal precision even in the presence of interactions.
  • In dummy-coded models (common in k-in-1 trials), separate effects and interactions are correlated, and the precision of separate effects degrades when interactions are present and included in the model.
• Decision Making: The authors contrast decision-making approaches, noting that DoE/MOST utilizes component screening or decision analysis for intervention value efficiency (DAIVE) to select optimal combinations, whereas k-in-1 trials typically focus on testing separate effects, often using two-stage testing procedures that the authors note can be misleading.

Results
Using the illustrative example of the smoking cessation trial:

• The data exhibited a significant antagonistic interaction (the combined effect of capability and opportunity text messages was worse than the sum of their individual effects).
• Additive Model (Dummy Coded): Estimated marginal effects that were non-significant and suggested a decrease in abstinence rates, failing to capture the conditional nature of the effects.
• Maximal Model (Dummy Coded): Estimated separate effects (capability alone, opportunity alone) which were positive but non-significant, with standard errors $\sqrt{2}$ times larger than the marginal effects. The separate interaction was significant.
• Effect-Coded Maximal Model: Estimated main effects and interactions with equal precision. The main effects were non-significant, and the interaction was significant.
• The analysis demonstrated that presenting marginal effects alone (as often done in k-in-1 analyses) is inadequate when interactions exist. The choice of model (additive vs. maximal) and coding (dummy vs. effect) fundamentally changes the estimated treatment effects and their precision. In the 2×2×2 case, the authors showed that effect-coded models maintain equal precision for all effects (main and interactions) as the number of factors increases, whereas dummy-coded models show increasing variance for higher-order separate interactions.

Significance and Claims
The paper claims that the confusion surrounding factorial trials stems from a lack of clarity regarding the trial's primary objective and the resulting estimands. The authors argue that:

• Blanket statements advising against factorial trials when interactions are plausible are incorrect; such advice is specific to k-in-1 trials aiming for efficiency in estimating separate effects.
• Factorial trials remain highly efficient for screening, optimizing, and evaluating complex interventions (main effects and interactions) even when interactions are present, provided the sample size is calculated based on the desired main effects and the analysis uses appropriate models (e.g., effect-coded maximal models).
• It is vital for trialists to specify their objectives, estimands, and treatment contrasts clearly at the design stage. The choice of design (e.g., 2-in-1 vs. factorial optimization) and analysis strategy must follow from these specifications.
• The authors suggest that clinical trials, particularly those involving complex interventions, should embrace methods long established in the Design of Experiments field, moving beyond the limitations of the "no interaction" assumption that has dominated UK clinical trial methodology. They conclude that the difficulties purported to render factorial designs inappropriate for most clinical trials are often specific to the k-in-1 context and do not apply to the broader application of factorial designs in health research.