💰 quantitative finance

A Multiscale Ball Test for Conditional Mean Independence

This paper introduces the Multiscale Ball Conditional Mean Independence (MBCMI) test, a robust statistical method that aggregates local mean contrasts across varying spatial scales to effectively detect conditional mean dependence in both independent and serially correlated data, particularly excelling at identifying localized and radial signal departures.

Original authors: Simon Rudkin, Wanling Rudkin

Published 2026-08-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Simon Rudkin, Wanling Rudkin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of data science, researchers often try to understand how one thing influences another. Imagine a weather forecaster trying to predict rain. They might look at temperature, humidity, and wind speed. If the average amount of rain on a given day depends on the temperature, then the two are linked. But what if the relationship is subtle? What if rain only happens when the temperature is high and the humidity is low, but only in a specific corner of the map? Standard tools for finding these links often look at the big picture, averaging out the details. This approach can miss small, localized patterns that exist only in a specific region of the data. It is like trying to find a small, hidden island by looking at a map of the entire ocean; the island might be there, but the average depth of the water makes it invisible.

This is the challenge Simon Rudkin and Wanling Rudkin set out to solve. They developed a new way to test whether one set of numbers influences another, specifically looking for relationships that hide in small, distinct areas rather than spreading evenly across the whole dataset. Their method, called the Multiscale Ball Conditional Mean Independence test, is designed to find these hidden connections without needing to know exactly where to look or how big the hidden area might be. By testing many different sizes of "search zones" at once, they created a tool that is sensitive to local changes, offering a sharper view of how variables interact in complex systems like financial markets.

The researchers built their test around a simple geometric idea: drawing balls around data points. In their method, every single observation in a dataset becomes the center of a ball. They then draw balls of many different sizes around each center, ranging from very small clusters of neighbors to large groups that cover a significant portion of the data. Inside each ball, they calculate the average outcome and compare it to the overall average for the entire dataset. If the average inside a specific ball is significantly different from the global average, it suggests a local relationship exists. The key innovation is that they do not just pick one size for these balls. Instead, they test a wide range of sizes, from the smallest clusters that contain just a few points up to large spheres that encompass nearly three-quarters of the data. They then look for the specific size that reveals the strongest difference, effectively letting the data tell them where the signal is hiding.

To ensure their findings were not just random noise, the authors tested their method against a variety of scenarios using computer simulations. They created artificial data with known patterns, including smooth global trends, sharp local islands of activity, and complex checkerboard patterns where positive and negative effects alternate rapidly. The results showed that their new test excels at finding "local islands" and "ring-shaped" signals, where a relationship exists only in a specific region or a specific distance from a center. However, they also found that their method is not a universal winner. When the data contains rapidly changing patterns, like a checkerboard where the effect flips sign from one point to the next, a different type of test based on nearest neighbors performed better. This distinction is crucial: the new tool is not a magic bullet that solves every problem, but a specialized instrument that is superior for finding spatially coherent, localized relationships.

The authors then applied their method to real-world financial data, examining monthly records from the United States spanning from January 1980 to September 2025. They looked at how stock market factors, such as market returns and momentum, relate to one another, and how macroeconomic variables like unemployment and industrial production interact with market returns. In their initial analysis, the test flagged several of these relationships as significant, suggesting that the average outcome for one variable changed depending on the state of the others. However, when they removed the standard, linear relationships that are already well-known in finance—essentially stripping away the "obvious" connections—the significant results largely disappeared. This finding suggests that the apparent non-linear patterns they detected were actually just reflections of the standard linear dependencies that economists already understand, rather than evidence of a new, mysterious type of market behavior.

There was one notable exception to this pattern. In a specific window of time during July 2022, the test found a significant relationship involving the momentum factor that remained even after the standard linear connections were removed. This suggests that during that specific period, there was a genuine, localized dependence in the data that went beyond the usual linear models. However, the authors are careful to note that this result comes from a specific time window selected after looking at the data, so it serves more as a descriptive observation than a confirmed, permanent rule. The study concludes that while the new test is powerful for detecting local, spatially coherent signals, it also reveals that many apparent anomalies in financial data are simply the result of standard linear relationships that were not fully accounted for. The tool provides a clearer lens for spotting where data behaves differently, but it also helps researchers distinguish between true structural changes and the noise of standard market dynamics.

Technical Summary: A Multiscale Ball Test for Conditional Mean Independence

Problem Statement
The paper addresses the challenge of testing conditional mean independence, defined as $H_0: E[Y | X] = E[Y]$ almost surely. While global conditional-moment procedures (e.g., Bierens, 1990; Fan and Li, 1996) integrate discrepancies over the entire predictor support, they can lose power when departures from the null are confined to a bounded region of the multivariate predictor space. Conversely, local smoothers avoid global averaging but face a tuning problem: the spatial scale at which a signal separates from noise is rarely known in advance. Existing methods often either fix a global integration scale or fix a local mass (e.g., nearest-neighbour counts), failing to balance local smoothing with robustness to noise when the relevant spatial scale is unknown.

Methodology: The Multiscale Ball Conditional Mean Independence (MBCMI) Test
The authors propose the MBCMI test, which aggregates support-weighted local mean contrasts across "balls" centered on each observed data point in the predictor set.

Population Criterion: The test targets a population functional $Q(e)$ , which averages squared local mean contrasts weighted by the probability mass supporting them. This weighting prevents small, weakly supported neighborhoods from dominating the statistic and aligns the sample statistic with the population local-mean criterion.
Sample Statistic: For a given radius $e$ , the statistic $T_n(e)$ sums the squared differences between the local mean within a ball centered at $X_i$ and the global mean, weighted by the number of observations in that ball ( $N_i(e)$ ).
Multiscale Search: Instead of selecting a single bandwidth, the test evaluates the statistic over a dense grid of 71 radii. These radii are defined by empirical pairwise-distance quantiles $q \in \{0.05, 0.06, \dots, 0.75\}$ . The final test statistic is the maximum of these 71 values.
Inference and Calibration:
- IID Data: The authors establish that the raw maximum over the fixed grid behaves as a finite collection of degenerate V-statistics. They prove conditional validity for a Rademacher multiplier bootstrap, which preserves observation-specific residual magnitudes.
- Serial Data: For time-series applications, a "prewhitened recursive Rademacher bootstrap" is proposed. This involves fitting a finite-order autoregression (order selected via BIC), randomizing the signs of the estimated innovations, and recursively reconstructing the series. This preserves the innovation magnitudes and the fitted dynamic law.
Diagnostics: The method provides a "centre-by-radius" decomposition, allowing researchers to identify which scales and specific predictor regions contribute most to the rejection, though these are diagnostic rather than structural parameters.

Key Contributions

Geometric Architecture: Unlike nearest-neighbour methods (e.g., Chatterjee et al., 2026) that fix local mass and let spatial extent vary, MBCMI fixes the spatial extent (radius) and lets local mass vary. This architecture is distinct from global integration or fixed-bandwidth smoothing.
Theoretical Framework: The paper provides fixed-grid theory identifying the population target and establishing consistency for alternatives "visible" at the indexed radii. A new Pitman local-power limit is derived, showing that local power depends on the Ball-smoothed mean departure at those specific radii.
Feasible Serial Validity: The authors establish the validity of the recursive-sign bootstrap for stable finite-order autoregressions with conditionally sign-symmetric innovations, a result that accounts for nuisance estimation and dependence structure.
Geometry-Specific Power Envelope: The paper demonstrates that no single test dominates universally. MBCMI is shown to be strongest for local and radial signals (e.g., local islands, annular departures), whereas nearest-neighbour graph methods excel for rapidly sign-changing interactions and checkerboard patterns.

Empirical Results

Calibration: In serial null experiments aligned with the application, the raw maximum rejects 4.25% of the time (within the 95% confidence interval of the nominal 5%), demonstrating feasible validity.
Power Analysis: In canonical Monte Carlo experiments ( $n=500, d=10$ ), MBCMI outperforms nearest-neighbour methods (NCMD) on local island, ring, thin band, and local absolute signal geometries. Conversely, NCMD ( $K=5, 10$ ) significantly outperforms MBCMI on interaction and checkerboard alternatives.
Robustness: Predictor-law experiments indicate that the method's strengths are not artifacts of independent Gaussian covariates. However, standardization choices (z-score vs. whitening) affect power, particularly under correlated predictors.
Finance Applications: Applied to U.S. monthly finance data (1980–2025), full-sample MBCMI tests rejected conditional mean independence for five of six Fama-French factors and the unemployment rate change. However, when applied to cross-fitted residuals (removing contemporaneous linear structure), none of the ten full-sample equations rejected. This suggests the detected dependence is contemporaneous conditional-mean dependence rather than distinctive nonlinear structure. A specific diagnostic window (Momentum in July 2022) remained significant after linear residualization.

Significance and Claims
The paper claims a "bounded methodological contribution." It does not assert universal power dominance or omnibus consistency against all non-constant conditional means. Instead, it offers a test specifically designed for scenarios where conditional-mean departures occupy spatially coherent regions at an unknown scale.

The authors emphasize that the test's value lies in the combination of a fixed-extent multiscale search and an interpretable centre-by-scale decomposition. The finance applications illustrate a critical distinction: the test can detect contemporaneous conditional-mean dependence, but without residualization, this does not imply the existence of nonlinear mechanisms or forecasting ability. The paper concludes that MBCMI provides a formal, assumption-specific tool for detecting local and radial signals, complementing rather than replacing nearest-neighbour or global integration approaches.

More like this