📊 statistics

Post-selection inference in generalized linear models via parametric programming

This paper proposes a unified parametric programming framework for post-selection inference in generalized linear models following Lasso variable selection, which adapts Gaussian-based strategies to non-Gaussian responses via a linearized pseudo-model to effectively correct naive inference while outperforming polyhedral-based adjustments in efficiency.

Original authors: Qinyan Shen, Karl Gregory, Xianzheng Huang

Published 2026-03-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qinyan Shen, Karl Gregory, Xianzheng Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery. You have a massive evidence board with 100 different clues (variables) pinned to it. Your goal is to figure out which 5 clues actually caused the crime (the outcome) and which 95 are just red herrings.

Here is the problem: If you look at all 100 clues, pick the 5 that look the most suspicious, and then immediately run a standard math test to say, "Aha! These 5 are definitely guilty!", you are likely to make a mistake. You might convict an innocent person just because they looked suspicious after you picked them out of the crowd. In statistics, this is called Post-Selection Inference. The "standard math test" (naive inference) doesn't know you already did some detective work to pick those 5, so it gets too confident and gives you false alarms.

This paper proposes a new, smarter way to do this detective work, specifically for situations where the "crime" isn't a simple number (like height or weight) but something trickier, like:

Yes/No answers (Logistic Regression: Will the email be spam?).
Counts (Poisson Regression: How many times did a patient visit the doctor?).
Percentages (Beta Regression: What percentage of the test did a student get right?).

Here is the paper's solution, broken down into three simple steps using a creative analogy.

The Problem: The "Magic 8-Ball" Trap

Traditional methods try to fix the "false alarm" problem by being extremely cautious. They say, "Okay, we picked these 5 suspects, but we also need to know the exact direction they were pointing (positive or negative) to be sure."

The authors call this the "Polyhedral Method." It's like a detective who, after picking the 5 suspects, insists on also checking if they were wearing red shoes, blue shoes, or no shoes at all before making a final arrest. This extra checking makes the detective so cautious that their "confidence intervals" (the range of guilt they are willing to accept) become huge. They might say, "This person is probably guilty, but they could also be innocent," which isn't very helpful. It's like a "Magic 8-Ball" that only says "Ask again later."

The Solution: The "Translator" Strategy

The authors propose a new method called Parametric Programming (PP) combined with Linearization. Think of this as a two-step translation process.

Step 1: The "Translator" (Linearization)

The paper deals with complex data types (like "Yes/No" or "Counts") that are hard to analyze directly.

The Analogy: Imagine you are trying to navigate a winding, mountainous road (the complex data) to get to a destination. It's hard to drive straight there.
The Fix: The authors say, "Let's build a straight, flat highway that looks exactly like the mountain road at the specific spot where we are standing."
How it works: They take the complex, non-linear data and create a "pseudo-response" (a fake, simplified version of the data) that behaves like a straight line. They call this Linearization. Now, instead of driving on a scary mountain, they are driving on a smooth, straight highway.

Step 2: The "Smart Detective" (Parametric Programming)

Now that they are on the straight highway (the linear model), they need to pick the guilty suspects without making the "Magic 8-Ball" mistake.

The Old Way (Polyhedral): As mentioned, the old method checks everything (including the direction of the clues), making the investigation slow and the results vague.
The New Way (Parametric Programming): The authors use a clever mathematical trick. Instead of checking every single detail of the suspects, they ask: "If I wiggle the data just a tiny bit, at what point does my list of 5 suspects change?"
The Analogy: Imagine your list of 5 suspects is written on a piece of paper. The new method asks, "How much can I shake the table before the paper falls off and I have to pick a different group of 5 people?"
The Result: This allows them to calculate the "guilt" of the suspects much more precisely. They don't need to check the shoe color (the extra signs); they just need to know how stable their list of suspects is. This results in tighter, more accurate confidence intervals.

Why This Matters (The Real-World Test)

The authors tested this on three real-world scenarios:

Spam Emails: They found that the old method claimed 19 words were "spam indicators," but the new method correctly identified that only 13 were truly significant. The old method was over-confident and flagged innocent words.
Doctor Visits: The old method said a student's school attendance was a major factor in doctor visits. The new method realized that once you account for health status, school attendance wasn't actually that important.
Student Grades: The old method thought "going out with friends" and "health" were huge factors in grades. The new method said, "Actually, those are weak factors; the real drivers are absences and failing previous classes."

The Bottom Line

This paper gives statisticians a new tool to analyze complex data (like yes/no, counts, or percentages) after they have already picked the most important variables.

Old Way: "We picked these variables, so we are 95% sure they matter," but the range of "sure" is so wide it's useless, or it's too narrow and leads to false arrests.
New Way: "We translated the complex data into a straight line, then used a smart math trick to see how stable our choices are. This gives us a much sharper, more accurate picture of what is actually causing the effect, without the false alarms."

It's like upgrading from a blurry, shaky camera to a high-definition lens that knows exactly how the photo was taken, so you can trust the picture you see.

1. Problem Statement

In regression analysis, practitioners often use data-driven methods (such as Lasso) to select a subset of covariates before performing statistical inference. Traditional inference methods (e.g., standard Wald tests) assume the model is fixed a priori. When applied to selected models, these "naive" methods ignore the selection event, leading to:

Inflated Type I error rates: False positives occur far more frequently than the nominal significance level.
Invalid confidence intervals: Coverage probabilities drop significantly below the target level.

While post-selection inference (PSI) methods exist for Gaussian linear models (e.g., the polyhedral method by Lee et al., 2016), extending these to Generalized Linear Models (GLMs) for non-Gaussian responses (e.g., binary, count, or proportion data) is challenging. Existing extensions often suffer from:

Over-conditioning: Conditioning on the signs of selected coefficients to make the selection event polyhedral, which reduces statistical efficiency and results in unnecessarily wide confidence intervals.
Inflexibility: Difficulty in accommodating data-driven selection of the tuning parameter ( $\lambda$ ).

2. Methodology

The authors propose a unified framework called Parametric Programming after Linearization (PPL). The methodology consists of two main stages:

Step 1: Model Linearization

The authors transform the non-Gaussian GLM problem into an approximate Gaussian linear regression problem.

Newton-Raphson/Fisher Scoring Connection: They leverage the iterative reweighted least squares (IRLS) formulation used to find Maximum Likelihood Estimators (MLEs) in GLMs.
Pseudo-Data Construction: Based on the converged MLE $(\hat{\beta}_0, \hat{\beta})$ $(\hat{β}_{0}, \hat{β})$ , they construct:
- A pseudo-response vector $\hat{z}_0$ (a "centered" version of the working response).
- A pseudo-design matrix $\hat{U}_0$ (centered design matrix).
Theoretical Justification: They prove that under regularity conditions, the conditional sampling distribution of the estimator in the original GLM is asymptotically equivalent to that of the least squares estimator in the linearized model using $\hat{z}_0$ and $\hat{U}_0$ . This allows the application of linear model PSI techniques to GLMs.

Step 2: Parametric Programming (PP) on the Linearized Model

Instead of using the polyhedral method (which conditions on signs), the authors apply the Parametric Programming (PP) approach (Le Duy and Takeuchi, 2021) to the linearized data.

Parameterization: The selection event (defined by the Lasso solution path) is parameterized by a scalar $\tau$ . The response vector is expressed as $z(\tau) = q + \tau \cdot c$ , where $c$ is the contrast vector of interest.
Selection Event Characterization: The set of $\tau$ values for which the Lasso selects the specific model $M$ is identified as a union of disjoint intervals ( $T_M$ ).
Conditional Distribution: The conditional distribution of the estimator is derived as a truncated normal distribution over the interval $T_M$ .
Handling Tuning Parameter Selection: The framework is extended to handle data-driven selection of $\lambda$ (e.g., via cross-validation or train/validation splits) by further parameterizing the selection of $\lambda$ within the PP framework.

3. Key Contributions

Unified Framework for Non-Gaussian GLMs: The paper successfully adapts the PP strategy to GLMs (Logistic, Poisson) and non-GLM settings (Beta regression) by establishing a rigorous link between MLE in GLMs and Least Squares in linear models.
Avoidance of Over-Conditioning: Unlike polyhedral methods that condition on the signs of coefficients (leading to efficiency loss), the PP method conditions only on the selection event. This results in narrower confidence intervals and higher statistical power, particularly for null effects where coefficient signs are unstable.
Data-Driven Tuning Parameter Selection: The method explicitly accounts for the selection of the penalty parameter $\lambda$ based on the data, a feature often missing in standard polyhedral approaches.
Theoretical Guarantees: The authors provide asymptotic proofs showing that the inference based on the observable pseudo-data is equivalent to inference based on idealized (unobservable) data, ensuring validity as $n \to \infty$ .

4. Results

The paper validates the method through extensive simulations and real-world case studies.

Simulation Studies

Type I Error Control: In Logistic, Poisson, and Beta regression settings, the PPL method maintains Type I error rates close to the nominal level (0.05) across various $\lambda$ values. In contrast, the naive method shows drastically inflated error rates (up to 0.35).
Efficiency vs. Polyhedral Method:
- Both methods control coverage probability (~95%).
- For truly non-zero coefficients, PPL intervals are slightly wider (due to less conditioning).
- For truly zero coefficients, PPL intervals are substantially tighter than polyhedral intervals. This demonstrates that the polyhedral method's conditioning on signs is detrimental to power when inferring null effects.
Robustness: The method performs well even when $\lambda$ is selected via a data-driven 70/30 split.

Real Data Applications

Spambase (Logistic Regression): PPL identified 13 significant features out of 25 selected, whereas the naive method identified 19. PPL correctly downgraded features (e.g., specific word frequencies) that were likely false positives in the naive analysis.
Medicaid Data (Poisson Regression): PPL identified 3 significant covariates out of 4 selected, while the naive method claimed all 4 were significant. PPL correctly identified "school" as insignificant, aligning with prior literature that suggests health status is the primary driver.
Student Performance (Beta Regression): PPL identified 10 significant variables out of 27, compared to 13 by the naive method. Notably, PPL correctly flagged "goout" and "health" as insignificant, consistent with literature suggesting these have weak predictive power compared to academic factors.

5. Significance

This paper addresses a critical gap in modern statistical inference: the ability to perform valid hypothesis testing after variable selection in non-Gaussian settings without sacrificing statistical efficiency.

Practical Impact: It provides a computationally feasible and theoretically sound tool for researchers working with binary, count, or proportion data who rely on Lasso for feature selection.
Methodological Advancement: By demonstrating that "linearization" followed by "parametric programming" is superior to "polyhedral conditioning," the authors offer a more efficient alternative that reduces the width of confidence intervals and increases the power to detect true effects while strictly controlling false positives.
Versatility: The framework's applicability to Beta regression (a non-GLM) suggests its potential for broader application in complex regression settings beyond standard exponential families.