📊 statistics

Retrospective Orthogonal Design: Response-Surface Reconstruction from Observational Data

This paper introduces Retrospective Orthogonal Design (ROD), a novel method that reconstructs conditional mean surfaces from observational data onto a probability-balanced lattice to achieve specification-invariant, order-independent regression estimates and superior out-of-sample performance on complex, non-linear surfaces compared to traditional polynomial regression.

Original authors: Lawrence Fulton, Christopher Fulton, Arvind Sharma, Aleksandar Tomic

Published 2026-07-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lawrence Fulton, Christopher Fulton, Arvind Sharma, Aleksandar Tomic

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to figure out what makes a cake taste good. You have a giant bowl of ingredients: flour, sugar, eggs, and chocolate. In a perfect science lab, you would mix them in exact, separate amounts to see how each one changes the flavor. This is called a "balanced design," and it makes the math easy because every ingredient is independent. But in the real world, ingredients often come pre-mixed. Maybe the flour you bought always has a little extra sugar in it, or the eggs are always slightly larger when you buy the chocolate. This is called "correlation." When you try to taste the cake to see which ingredient is the star, the math gets messy. You can't tell if the sweetness comes from the sugar or the extra sugar in the flour.

This is the problem statisticians face with "observational data." It's data we collect from the real world—like how much money people make based on their schooling and experience—rather than data from a controlled experiment. Because real-life factors are tangled together, traditional math tools often give different answers depending on the order you ask the questions. If you ask about schooling first, it gets all the credit. If you ask about experience first, it gets the credit. It's like a game where the winner changes just because you changed the rules of who goes first. Scientists care about this because they want to know the true cause of things, not just a result that depends on how they arranged their calculator. They need a way to untangle the ingredients without throwing away the cake.

Enter a new method called Retrospective Orthogonal Design (ROD), created by Lawrence Fulton, Christopher Fulton, Arvind Sharma, and Aleksandar Tomić. Think of ROD as a magical, high-tech kitchen sieve that forces messy, pre-mixed ingredients into a perfect, orderly grid after you've already tasted the cake.

Here is how it works: Imagine you have a messy pile of cake batter samples with different amounts of flour and sugar. Instead of trying to guess the recipe from the mess, ROD builds a perfect, invisible grid over the batter. It divides the world of "flour" and "sugar" into equal-sized, perfectly balanced boxes. If a box has batter in it, ROD measures the average taste of that box. If a box is empty (because no one happened to bake a cake with exactly those amounts), ROD uses a clever "neighbor" trick to guess what the taste would have been, based on the boxes right next to it.

Once the grid is full, ROD does something amazing: it rearranges the math so that every box in the grid is perfectly independent of the others. It's like taking a tangled ball of yarn and magically straightening every single strand so they don't touch. Because the grid is now perfectly balanced, the math becomes "order-invariant." This means it doesn't matter if you ask about flour first or sugar first; the answer is always the same. The credit for the sweetness is split fairly and exactly, no matter how you look at it.

The authors tested this idea using a massive computer simulation with over 6,000 different scenarios. They created fake worlds with different rules—some where the relationship was a simple straight line, some where it was a bumpy curve, and some where it was a sharp "on/off" switch. They found that ROD was a champion at handling the tricky, bumpy, and switch-like worlds where traditional methods struggled. In these cases, ROD was much better at predicting the outcome and gave a much clearer picture of what was driving the changes. For the simple, straight-line worlds, ROD was just as good as the old methods, but not necessarily better.

The paper also shows that ROD can translate its "grid" answers back into the language of standard formulas, so scientists can still use familiar terms like "years of schooling" or "years of experience" without losing the fairness of the new method. In a real-world test using data on how education affects earnings (the famous Mincer equation), ROD produced predictions that were just as accurate as the best traditional models, but it provided a single, unshakeable breakdown of how much credit education versus experience deserved, regardless of the order of calculation.

The authors are careful to say that ROD isn't a magic wand that solves every problem or proves why something happens (like whether more schooling causes higher pay, or if smart people just happen to stay in school longer). It doesn't create new data out of thin air. But it does offer a way to look at messy, real-world data that is fair, consistent, and reproducible. It turns the question from "What happens if I change the order of my questions?" into "Here is exactly how the variation is distributed on this specific, balanced map." For anyone trying to make sense of a tangled world, ROD offers a way to see the pattern clearly, one perfectly balanced step at a time.

Technical Summary: Retrospective Orthogonal Design (ROD)

Problem Statement
In observational studies, regression estimates and variance decompositions are often unstable due to multicollinearity among predictors. Unlike balanced experimental designs where $X^\top X$ is diagonal, observational data yields non-diagonal Gram matrices, causing coefficient estimates, unique variance attribution, and sequential sums of squares (SS) to depend heavily on model specification and the order of term entry. Existing remedies, such as Principal Components Regression (PCR) or orthogonal polynomial contrasts, often sacrifice interpretability in the original variable space or fail to address cross-predictor dependence. Furthermore, nonparametric methods (e.g., regression trees, splines) lack exact empirical orthogonality, making global attribution dependent on partitioning or smoothing choices. The authors posit that current methods fail to reconstruct an interpretable, orthogonal response surface that preserves observed data structure while providing specification-invariant attribution.

Methodology: Retrospective Orthogonal Design (ROD)
ROD reframes the analysis of observational data as a surface reconstruction problem on a probability-balanced lattice. The workflow consists of eight steps:

Predictor Partitioning: Predictors are divided into "grid predictors" (used for lattice construction) and "block predictors" (handled via standard coding).
Probability-Scale Remapping: Grid predictors are transformed to a probability scale ( $r_j = F_j(x_j)$ ) and then to a reference scale ( $z_j = 2a r_j - a$ ) to balance marginal occupancy and reduce sparse tails, without assuming a specific distribution.
Lattice Resolution Selection: A lattice resolution $L$ is selected via cross-validation among rank-admissible candidates. The grid creates $L^k$ cells.
Cell Assignment and Normalization: Observations are assigned to grid cells. Weights are normalized so that each cell has an equal target total weight, establishing a balanced product reference measure.
Cell Identification: Occupied cells retain their weighted observed means. Empty (unsupported) cells are identified for completion.
Lattice Completion: Empty cells are filled using a selected rule (e.g., global main-effects WLS or local inverse-distance weighting) while preserving the means of occupied cells.
Reconstruction and Evaluation: An orthogonal contrast matrix $X_L$ is constructed on the completed lattice. Coefficients are estimated as $\hat{\beta}_L = c_L^{-1} X_L^\top W_L \tilde{y}_L$ , where $X_L^\top W_L X_L = c_L I$ . The surface is evaluated via piecewise-affine interpolation over Freudenthal polyhedra, ensuring continuity and exactness at lattice vertices.
Calibration and Projection: If a structural basis (e.g., a specific polynomial form) is declared, the reconstructed surface is projected onto it. A "response-free" calibration matrix, derived solely from predictor geometry and the reconstruction pipeline, corrects for finite-resolution recovery loss, mapping the native orthogonal surface to the declared scientific basis.

Theoretical Properties
The paper establishes that for an admissible lattice, the weighted Gram matrix satisfies $X^\top W X = cI$ . This yields:

Algebraic Orthogonality: Exact orthogonality under the declared reference measure.
Specification Invariance: Contrast effects and coefficients are invariant to the inclusion of other orthogonal terms.
Order-Independent SS: Type I, II, and III sums of squares coincide within the retained contrast space, eliminating the dependence on term-entry order.
Dual Identity: The method functions simultaneously as an orthogonal contrast system and a reconstructed response surface.

Key Results
The authors evaluated ROD across 6,480 simulation conditions spanning nine data-generating processes (DGPs), varying sample sizes, predictor dimensions, and correlation levels.

Predictive Performance: ROD matched or exceeded polynomial regression (POLY) in five of nine processes. It performed exceptionally well on surfaces with thresholds, sign-interactions (XOR), and localized bumps, outperforming POLY by mean $\Delta R^2$ of 0.22, 0.32, and 0.09, respectively. For linear and quadratic-interaction processes, ROD was practically indistinguishable from POLY (mean $\Delta R^2 \approx 0.0001$ ). However, ROD underperformed on smoother or distributionally complex processes (e.g., exponential growth, hurdle models).
Coefficient Recovery: Calibrated ROD projections onto prespecified structural bases showed near-zero bias and stable dispersion, outperforming parametric models in high-dimensional settings where parametric bias increased with dimension.
Empirical Application: In a weighted Mincer earnings equation analysis (CPS data), ROD achieved the highest out-of-sample $R^2$ point estimate (0.1824), closely overlapping with POLY (0.1799). Crucially, ROD provided a single, exhaustive, and order-invariant allocation of sums of squares, whereas OLS and POLY allocations varied significantly across the 6 and 120 possible term-entry orderings, respectively.

Significance and Claims
The paper claims that ROD's primary contribution is interpretive stability rather than universal predictive superiority. By constructing an admissible, balanced reference surface a priori, ROD makes attribution a reproducible property of the reconstruction rather than a consequence of modeling choices or term ordering.

Attribution: ROD allows analysts to decompose variance into unique, non-overlapping shares that are invariant to specification order, addressing a fundamental limitation of regression with correlated predictors.
Transparency: The method explicitly handles incomplete support through completion rules, making assumptions about unobserved regions visible and auditable.
Limitations: The authors emphasize that ROD is representational, not causal; it does not resolve identification issues like ability bias. Its orthogonality is conditional on the declared reference measure and the chosen completion rule. The method is most valuable when the research goal requires stable, comparable, and traceable explanations of variation in the presence of nonlinearities and multicollinearity, rather than when prediction is the sole objective or a low-dimensional parametric model is sufficient.

The authors provide an operational Python implementation (ROD 0.4) to facilitate the reconstruction, validation, and calibration procedures described.

Technical Summary: Retrospective Orthogonal Design (ROD)

More like this