Least trimmed squares regression with missing values and cellwise outliers

This paper proposes a new least trimmed squares regression method that simultaneously handles missing values, resists both casewise and cellwise outliers, accommodates skewed distributions, and enables robust out-of-sample predictions.

The Gist

Imagine you are trying to bake the perfect cake based on a recipe that uses 20 different ingredients. You have data from 3,000 different bakeries. Most of them follow the recipe perfectly, but some have made mistakes.

In the world of statistics, this is called Regression. It's the tool we use to find the "recipe" (the relationship between ingredients and the final cake) that works best for the majority of the data.

However, real-world data is messy. This paper introduces a new, super-smart way to handle two specific types of mess:

1. The "Bad Batch" (Casewise Outliers): An entire bakery messed up the whole cake (maybe they forgot the oven).
2. The "Sour Spoon" (Cellwise Outliers): A bakery got the recipe mostly right, but one specific ingredient was measured wrong (e.g., they used salt instead of sugar).

The Problem with Old Methods

Traditional methods (like Ordinary Least Squares) are like a chef who tastes the whole batch and tries to average it out. If one bakery put in a cup of salt instead of sugar, the chef might think, "Oh, maybe the recipe actually needs salt!" and ruin the recipe for everyone else.

Even newer "robust" methods are good at spotting the "Bad Batches" (entirely wrong bakeries), but they often get confused by the "Sour Spoons." If one ingredient is wrong, they might throw away the whole bakery's data, or worse, they might not know how to predict the cake for a new bakery that has a wrong ingredient.

The New Solution: "CellLTS"

The authors, Jakob and Peter, propose a new method called CellLTS. Think of it as a two-step detective process that acts like a very careful, paranoid chef.

Step 1: The "Ingredient Scrub" (Cleaning the Predictors)

Before even looking at the final cake (the result), the method looks at the ingredients (the predictors) first.

• The Symmetrization Trick: Imagine you have a list of heights. Some are very tall, some very short. To make the math easier and fairer, the method creates a "mirror world." It pairs every person with every other person and looks at the difference between them. This turns a lopsided, messy list into a neat, symmetrical bell curve. It's like taking a jagged rock and grinding it down until it's a smooth, perfect sphere.
• The "CellMCD" Detective: Now, using this smooth data, the method scans every single ingredient. If a bakery says they used 500 pounds of flour (when the average is 5), the system flags that specific cell as "suspicious."
• The Repair: Instead of throwing the whole bakery away, the system says, "Okay, the flour measurement is wrong. Based on the other ingredients they used (like sugar and eggs), what should the flour amount have been?" It fills in the missing or wrong data with a "best guess" (imputation).

Step 2: The "Cake Tasting" (Robust Regression)

Now that the ingredient lists are cleaned and fixed, the method looks at the final cakes.

• The "Trimmed" Taste Test: It uses a technique called Least Trimmed Squares (LTS). Imagine tasting 100 cakes. Instead of averaging all 100 (which would be ruined by 5 burnt ones), the chef ignores the 25 worst-tasting cakes and calculates the average of the best 75. This finds the true "recipe" without being swayed by the disasters.

Why This is a Game-Changer: The "New Bakery" Test

The coolest part of this paper is how it handles Out-of-Sample Prediction.

Imagine a brand new bakery comes in. They have a list of ingredients, but one is missing, and one looks suspiciously high.

• Old Methods: Would say, "I can't use this data," or "I'll just plug these numbers into my formula," which would give a terrible prediction because the input was broken.
• CellLTS: Says, "Hold on. That ingredient looks weird. Let me check my 'Sour Spoon' database. It looks like a measurement error. I'll fix that number first, fill in the missing one, and then predict the cake."

It treats the new data with the same care as the old data, cleaning it before making a guess.

The Real-World Test: Cancer Rates

The authors tested this on real data about cancer death rates across US counties.

• The Mess: They found counties with impossible data, like a median age of 400 years (clearly a typo) or a cancer incidence rate that was way too high.
• The Result: The old methods got confused by these typos and gave weird predictions. CellLTS spotted the "400-year-old" error, fixed it, and gave a much more accurate prediction of cancer rates based on income, education, and age.

The Bottom Line

This paper gives statisticians a new, powerful tool that doesn't just ignore bad data; it fixes it.

• It handles missing values (like a missing ingredient).
• It spots single bad numbers (a sour spoon) without throwing away the whole recipe.
• It works even when the data is skewed (like a list of billionaires and regular people).
• It can predict the future for new, messy data by cleaning it first.

In short, it's a robust, self-correcting system that ensures your statistical "recipe" stays delicious, even when the kitchen is a bit of a disaster.

Technical Summary

Here is a detailed technical summary of the paper "Least trimmed squares regression with missing values and cellwise outliers" by Jakob Raymaekers and Peter J. Rousseeuw.

1. Problem Statement

Regression analysis is frequently hindered by outliers in real-world data. Traditional robust regression methods (e.g., Least Trimmed Squares - LTS, S-estimators, MM-estimators) primarily address casewise outliers, where an entire observation (row) is corrupted or belongs to a different population. However, modern datasets often contain cellwise outliers, where only specific entries (cells) within the data matrix (predictors or response) are suspicious or erroneous.

Existing cellwise robust regression methods (e.g., 3SGS, Shooting S, CR-Lasso) have significant limitations:

• They often rely on strong assumptions (e.g., Gaussianity).
• They generally fail to provide robust out-of-sample predictions. Most methods assume new data is free of cellwise errors, which is unrealistic in practice.
• They struggle with missing values and skewed distributions.

The paper addresses the need for a regression method that simultaneously handles:

1. Casewise outliers (entire rows).
2. Cellwise outliers (individual entries).
3. Missing values.
4. Skewed data distributions.
5. Robust out-of-sample prediction.

2. Methodology: The CellLTS Estimator

The authors propose a two-step method called CellLTS (Cellwise Least Trimmed Squares). The core innovation is a symmetrization strategy combined with a cleaning-imputation-regression pipeline.

Step 1: Cleaning and Imputing Regressors

The first step focuses on the predictor matrix $X$ without using the response variable $Y$.

• Symmetrization: To handle skewness and improve the performance of robust estimators, the data is symmetrized. For a sample $X$, the authors construct a dataset of pairwise differences: $Sym(X) = \{x_i - x_j\}$. This transforms the distribution closer to Gaussianity and removes the intercept term.
• Cellwise MCD (CellMCD): The authors apply the Cellwise Minimum Covariance Determinant (CellMCD) estimator to the symmetrized data. This estimates the location ($\mu$) and scatter matrix ($\Sigma$) of the clean data while flagging outlying cells.
• Imputation: Cells flagged as outliers (or missing) in the original $X$ are imputed using the best linear prediction based on the robust estimates of $\mu$ and $\Sigma$. This yields a "cleaned" regressor matrix $\hat{X}$.

Step 2: Robust Regression

The second step regresses the response $Y$ on the cleaned predictors $\hat{X}$.

• Symmetrization of Response: The response $Y$ is also symmetrized ($Sym(Y)$) and standardized.
• LTS with Penalty: A Least Trimmed Squares (LTS) regression is performed on the symmetrized data. The objective function minimizes the sum of the smallest squared residuals over a subset of size $h$ (typically $0.75n$).
• Ridge Penalty: A small L2 penalty ($\lambda ||\beta||^2_2$) is added to the objective function to handle collinearity. This penalty is integrated into the LTS optimization via an augmented dataset approach (modifying the FastLTS algorithm).
• Intercept Estimation: The intercept for the original (unsymmetrized) model is estimated by applying a robust univariate location estimator (MCD) to the pseudo-residuals of the cleaned data.

Out-of-Sample Prediction

A key feature of CellLTS is its ability to predict for new data points ($x^*$) that may contain missing values or cellwise outliers:

1. Flagging: The new point $x^*$ is compared against the robust estimates ($\hat{\mu}, \hat{\Sigma}$) derived in Step 1. Cells exceeding a threshold are flagged.
2. Imputation: Flagged or missing cells in $x^*$ are imputed using the same linear prediction rule used in Step 1.
3. Prediction: The cleaned $x^*$ is multiplied by the robustly estimated coefficients. This ensures the prediction is not skewed by erroneous input values.

3. Key Contributions

• First Robust Out-of-Sample Prediction: Unlike previous cellwise methods, CellLTS provides a mechanism to clean and predict on new, potentially corrupted data.
• Breakdown Value Analysis: The authors prove the finite-sample breakdown value of CellLTS. They show that the method can withstand up to approximately 29% of contaminated cells per variable ($1 - 1/\sqrt{2}$). This is the first theoretical breakdown result for a cellwise robust regression method.
• Handling Skewness: By utilizing data symmetrization (pairwise differences), the method effectively handles skewed distributions without requiring complex transformations.
• Unified Framework: It seamlessly integrates missing value imputation, cellwise outlier detection, and casewise robust regression into a single algorithm.

4. Results

Simulation Study

The authors compared CellLTS against OLS, 3SGS, Shooting S, and STMW (CR-Lasso) under various conditions:

• Settings: Sample sizes $n=100, 400$; dimensions $d=10, 20$; contamination levels $\epsilon=10\%, 20\%$; and distributions (Gaussian, Exponential, Lognormal).
• Coefficient Accuracy: CellLTS maintained low Mean Distance (MD) errors for regression coefficients even as the magnitude of outliers ($\gamma$) increased, whereas competitors (especially 3SGS and Shooting S) showed significant degradation.
• Prediction Accuracy: CellLTS significantly outperformed all competitors in Mean Squared Error (MSE) for out-of-sample predictions. This is attributed to its ability to clean the new input data before prediction.
• Symmetrization Efficiency: The study confirmed that using a subset of random pairwise differences ($k=20$ permutations) yields results nearly identical to using all $O(n^2)$ pairs, making the method computationally feasible for larger datasets.

Real Data Application: US Cancer Mortality

The method was applied to a dataset of 3,047 US counties with 33 variables.

• Comparison: CellLTS identified different coefficient estimates compared to OLS, particularly for variables like median age, where OLS was heavily influenced by extreme outliers (e.g., counties with reported median ages of 400+ years).
• Diagnostic Power: The "Cellmap" visualization revealed specific data errors (e.g., impossible age values, incorrect cancer incidence rates in specific counties) that OLS missed.
• Imputation: The method successfully handled missing values (e.g., in employment rates) without prior preprocessing.

5. Significance

This paper represents a significant advancement in robust statistics by bridging the gap between cellwise and casewise robustness.

• Practical Utility: It solves a critical practical problem: how to make reliable predictions when new data is messy, incomplete, or contains specific entry errors.
• Theoretical Foundation: It establishes the first breakdown point theory for cellwise regression, providing a mathematical guarantee of robustness.
• Algorithmic Innovation: The combination of symmetrization, CellMCD, and penalized LTS creates a flexible tool that is less sensitive to distributional assumptions (like normality) than previous cellwise methods.

The authors provide open-source R code, making the method accessible for practitioners dealing with dirty, high-dimensional real-world data.