Technical Summary: Experimental Design When N Equals One

Problem Statement

N-of-1 trials (or time-series experiments) involve repeated randomization of treatment assignments for a single experimental unit over time. While widely used in clinical research and online platforms, the theoretically optimal design for estimating various treatment effects remains unclear. A central difficulty is that different target estimands (e.g., lag-specific effects vs. cumulative effects) favor different forms of temporal dependence in the treatment assignment path. Existing literature often relies on specific design-based approaches or independent assignments, but a general framework that flexibly targets different effects while providing optimality guarantees is lacking. Furthermore, optimal designs for specific estimands, particularly classes of estimands, are poorly characterized.

Methodology

The authors propose a Markovian framework for experimental design where the treatment assignment path $Z_1, \dots, Z_T$ is modeled as a Markov chain governed by time-varying transition matrices $P_t$. The states correspond to control ($0$) and treatment ($1$), with transition probabilities defined by parameters $\rho_t$ (probability of switching from 0 to 1) and $\gamma_t$ (probability of switching from 1 to 0).

To define the design objective, the paper adopts a finite-order impulse-response model:
$$Y_t = \alpha + \sum_{k=1}^K \beta_k Z_{t-k+1} + \epsilon_t$$
where outcomes depend additively on current and past $K-1$ treatment assignments. The target estimand is a linear combination of the impulse-response coefficients, $\tau_w = \sum w_k \beta_k$. The design objective is to minimize the asymptotic estimation variance of the Ordinary Least Squares (OLS) estimator for $\tau_w$.

The authors analyze two structured subclasses of Markovian designs to derive theoretical insights:

1. Random-Switch Designs: Characterized by constant transition probabilities ($\rho_t = \rho, \gamma_t = \gamma$). The design space is reduced to optimizing the switching probabilities.
2. Cycle-Switch Designs: Deterministic designs where the treatment status switches every $l$ time points. The design space is reduced to optimizing the block length $l$.

The optimization is framed as a minimax problem (robust design) or a targeted problem, minimizing the worst-case or specific estimation variance over a set of candidate designs and estimands. Theoretical analysis relies on large-$T$ asymptotics, utilizing the properties of Kac–Murdock–Szegő (KMS) matrices and ergodic theorems for Markov chains.

Key Contributions and Results

1. General Design Framework

The paper establishes a unified framework that encompasses existing designs (independent assignments, regular switchbacks, deterministic alternation) as special cases of Markovian assignment. It provides a principled method to control temporal dependence through transition probabilities.

2. Optimal Switching Probabilities (Random-Switch Designs)

For targeted design optimization (minimizing variance for a specific $w$):

• Lag-specific effects: The optimal design converges to $\rho = \gamma = 1/2$, corresponding to i.i.d. Bernoulli(1/2) randomization.
• Cumulative effects: The optimal switching probability moves to the boundary ($\rho, \gamma \to 0$), implying long, uninterrupted periods of treatment and control. However, this leads to ill-behaved limiting designs where OLS may be undefined.
• General effects: The optimal switching probability is determined by a quadratic equation involving the estimand vector $w$, balancing the trade-off between marginal variation and temporal persistence.

For robust design optimization (minimizing the worst-case variance over all normalized linear combinations of lag effects):

• The optimal design is i.i.d. Bernoulli(1/2) randomization ($\rho = \gamma = 1/2$). This provides a formal justification for the robustness of independent assignments in N-of-1 trials, consistent with findings in cross-sectional settings.

3. Optimal Block Lengths (Cycle-Switch Designs)

To address the boundary cases where random-switch designs fail (e.g., cumulative effects):

• Cumulative effects: The optimal block length is approximately $l^* \approx K + \sqrt{K}$. This balances the need for long periods (to capture cumulative effects) against the variance inflation caused by large blocks.
• Lag-specific effects: The optimal block length is $l^* = K$. This ensures identifiability of all lag-specific effects while minimizing variance.
• Robust designs: The optimal block length is $l^* = K$. Shorter blocks lead to rank-deficient covariance matrices (non-identifiability), while longer blocks increase variance linearly.

4. Simulation and Robustness

Simulations demonstrate that:

• Random-switch designs achieve the smallest estimation variance for lag-specific effects and remain robust across various model specifications, including model misspecification (e.g., unmodeled time trends).
• Cycle-switch designs outperform other designs for estimating cumulative effects in finite samples but exhibit higher bias under certain misspecifications (e.g., unmodeled time trends).
• The proposed designs generally outperform state-of-the-art alternatives like optimal regular switchback experiments (ORSB) when the target estimand is not purely carryover-focused.

5. Connection to Design-Based Inference

The paper connects the model-based OLS estimator to design-based inference. It shows that under random-switch designs, the OLS estimator converges to a linear projection of conditional-outcome contrasts. Under i.i.d. Bernoulli randomization or linear additive potential outcomes, this limit coincides with the lag-specific treatment effect estimand studied by Lin and Ding [2025], validating the causal interpretation of the estimator even without strict adherence to the linear impulse-response model.

Significance and Claims

The authors claim that their work provides a general design framework that can flexibly target different treatment effects while offering optimality guarantees. By characterizing the optimal temporal structure through switching probabilities and block lengths, the paper justifies the robustness of i.i.d. Bernoulli designs for lag-specific and robust objectives, while quantifying how optimal designs must deviate (e.g., longer blocks) for cumulative effects.

The paper emphasizes a trade-off between generality and interpretability: by adopting a model-based approach (impulse-response), they sacrifice some generality compared to nonparametric design-based methods but gain a unified framework for design optimization and clearer interpretability of the estimands. The results offer practical guidance for experimenters to select designs tailored to their specific scientific objectives (e.g., choosing cycle-switch designs for cumulative effects and random-switch designs for lag-specific effects).

The authors modestly note that while their framework is model-based, it connects to design-based inference under specific conditions. They also identify extending the framework to panel experiments (multiple experimental units) as a promising direction for future work.