📊 statistics

Zero-Inflated Logistic Regression Models with Shared Design: Identifiability, Existence of Estimates, and a Relabeling Rule

This paper establishes the identifiability and existence of maximum likelihood estimates for zero-inflated logistic regression models with shared design matrices, demonstrates the risks of ignoring zero-inflation mechanisms, and proposes a relabeling rule to resolve parameter symmetry for practical application.

Original authors: Yui Tomo, Shinto Eguchi, Daisuke Yoneoka

Published 2026-04-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yui Tomo, Shinto Eguchi, Daisuke Yoneoka

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Why are there so many "zeros" in your data?

In the world of statistics, a "zero" usually means "nothing happened." But sometimes, you get too many zeros. Maybe you are studying a disease, and many people say they don't have it. But are they truly healthy, or are they sick but didn't get tested? Or maybe they are sick but the test failed to catch it?

This is where the Zero-Inflated Logistic Regression model comes in. It's a special tool that assumes your data comes from two different groups mixed together:

The "Always Zero" Group: People who can never have the event (e.g., they are immune, or the event is impossible for them).
The "Maybe" Group: People who could have the event, but sometimes it just doesn't happen (e.g., they are susceptible, but the disease didn't strike this time).

The paper by Tomo, Eguchi, and Yoneoka tackles a very specific, tricky problem that happens when we use this tool in the real world. Here is the story of their discovery, explained simply.

1. The "Shared Design" Problem: The Twin Confusion

Usually, to solve the mystery, you need two different sets of clues (covariates) for the two groups. For example, you might use "age" to guess who is in the "Always Zero" group and "diet" to guess who is in the "Maybe" group.

But often, we don't know which clues matter for which group. So, we get lazy (or practical) and use the same list of clues for both groups. We use "age" and "diet" to guess both the "Always Zero" status and the "Maybe" status.

The Analogy: Imagine you have two identical twins, Bob and Charlie. You want to know which one is the "Always Zero" twin and which one is the "Maybe" twin. But you are only allowed to ask them the exact same questions.

If Bob says "Yes" to a question, and Charlie says "No," you can tell them apart.
But if they are mathematically identical in how they answer, the model gets confused. It can't tell if Bob is the "Always Zero" twin or if Charlie is the "Always Zero" twin.

In statistics, this is called Non-Identifiability. The model has two perfect solutions that look exactly the same to the math, just swapped. It's like looking in a mirror; you can't tell which side is the real you and which is the reflection.

2. The "Sign Flip" Trap: The Backwards Compass

The authors first warn us about what happens if we ignore this "Two-Group" mystery and just use a standard, simple model.

The Analogy: Imagine you are trying to navigate a forest.

The Truth: A specific tree (a variable) actually points you North (positive effect).
The Mistake: But there is a hidden fog (the "Always Zero" group) that hides the tree. If you ignore the fog and just look at the path, the tree might appear to point South (negative effect).

The paper proves mathematically that if you ignore the "Zero-Inflation" (the fog), your compass can spin 180 degrees. You might conclude that a risk factor protects you, when in reality, it harms you. This is a Sign Flip. It's a dangerous error that could lead to bad decisions.

3. The "Double Separation" Wall: The Dead End

In statistics, sometimes data is so perfectly separated that the math breaks down.

Standard Case: If you can draw a line that separates all "Yes" answers from all "No" answers perfectly, the math goes crazy (infinite estimates).
This Paper's Case: The authors found a "Double Separation." Imagine a wall that separates the "Yes" group from the "No" group in two different ways at the same time. If this happens, the model hits a dead end and cannot find a solution. They defined rules to tell you when this wall exists so you don't waste time trying to solve an unsolvable puzzle.

4. The Solution: The "Relabeling Rule"

So, we have a model with two identical twins (Bob and Charlie) that we can't tell apart, and the math says the solution is unique only if we stop caring who is who.

The Problem: In the real world, doctors and policymakers need to know who is who. They need to know: "Is this variable affecting the susceptible people or the immune people?"

The Creative Fix: The authors propose a simple Relabeling Rule.
Think of it like this:

First, run a simple, standard test (ignoring the two groups) to get a "rough guess" of what the "Maybe" group looks like. Let's call this the Reference Map.
Next, run the complex "Two-Group" model. It will give you two answers: (Bob=Group A, Charlie=Group B) OR (Bob=Group B, Charlie=Group A).
The Rule: Look at your two answers. Which one looks more like your Reference Map?
- If the "Bob=Group A" version looks like the Reference Map, then Bob is the "Maybe" group.
- If the "Bob=Group B" version looks like the Reference Map, then Bob is the "Always Zero" group.

It's like having a blurry photo of a suspect (the simple model). When the police find two identical twins, they compare the twins to the blurry photo. The twin who looks more like the photo is the one they arrest. It's not a perfect scientific proof, but it's a practical way to make a decision.

5. The Proof: The "Mirror Room" Experiment

To prove their theory, the authors ran computer simulations. They created "Mirror Rooms" (datasets) with different types of clues:

Continuous Clues (Smooth): Like height or weight. In these rooms, the two solutions (Bob/Charlie) were clearly separated into two distinct islands. The math worked perfectly.
Binary Clues (On/Off): Like "Yes/No" questions. In these rooms, the two solutions got jumbled together, and the model struggled to tell them apart.

This confirmed that if your data is too simple (only Yes/No answers), you might not be able to solve the mystery. But if you have enough variety in your data, the "Mirror Room" has a clear structure, and the model works.

Summary: What Should You Take Away?

Don't ignore the zeros: If you have too many zeros, a standard model might lie to you and flip your results upside down.
The "Shared" trap is real: If you use the same clues for both parts of the model, you can't mathematically tell the parts apart. The model has a "mirror symmetry."
But it's solvable: The model isn't broken; it just has a specific symmetry. If you accept that the two parts are interchangeable, the math is solid.
Use the "Relabeling Rule": To get a practical answer for real life, compare the complex model's output to a simple model's output. Whichever matches the simple model is likely the correct "susceptible" group.

In short, this paper gives us the rules of the road for driving a very powerful but tricky statistical car. It tells us where the potholes are (sign flips), how to handle the confusing roundabouts (symmetry), and gives us a GPS (the relabeling rule) to get to our destination.

1. Problem Statement

The paper addresses the Zero-Inflated Logistic Regression (ZILR) model, which is used for binary outcomes with an excess of zeros arising from a latent mixture of "susceptible" (event can occur) and "insusceptible" (structural zero) subpopulations, or from asymmetric misclassification.

The core problem investigated is the shared-design setting, where the same set of covariates (design matrix) is used for both components of the model:

The Ordinary Logistic Regression (OLR) component (modeling the event occurrence for susceptible individuals).
The Structural-Zero component (modeling the probability of being in the zero-inflation state).

Key Challenges:

Non-Identifiability: When covariates are shared, the likelihood function becomes invariant under the exchange of the two coefficient vectors ( $\beta$ and $\gamma$ ). This creates a symmetry where $(\beta, \gamma)$ and $(\gamma, \beta)$ yield identical likelihoods, making the ordered pair unidentifiable.
Estimation Failure: Standard optimization may fail to converge due to "separation" issues specific to the zero-inflated structure, or get trapped in symmetric local modes.
Misspecification Bias: Ignoring the zero-inflation mechanism when fitting a standard logistic regression can lead to severe bias, specifically a sign flip in the estimated coefficients.

2. Methodology and Theoretical Framework

A. Model Definition

The model defines the probability of observing $y_i=1$ as:
$P(y_i=1 | x_i) = F(x_i^\top \gamma) \cdot F(x_i^\top \beta)$
where $F(\cdot)$ is the inverse logit function. Here, $\beta$ governs the event occurrence (conditional on susceptibility), and $\gamma$ governs the susceptibility (structural zero state).

B. Theoretical Contributions

The authors establish four main theoretical pillars:

Sign-Flip Phenomenon under Misspecification:
- The authors prove that if a standard logistic regression is fitted to data generated by a ZILR model where a covariate has a positive effect on the event but a negative effect on the zero-inflation state (or vice versa), the resulting "pseudo-true" coefficient will have the opposite sign of the true underlying coefficient. This highlights the danger of ignoring zero-inflation structures.
Existence of Maximum Likelihood Estimates (MLE):
- The paper introduces the concept of "Double Separation" (analogous to separation in standard logistic regression). If the data satisfies double separation, the log-likelihood has no maximizer.
- Conversely, they define " $\epsilon$ -Double-Non-Separation" as a sufficient condition for the existence of an MLE. This ensures the data maintains a uniform margin preventing the likelihood from diverging.
Identifiability under Shared Design:
- While the ordered pair $(\beta, \gamma)$ is not identifiable, the authors prove that the model is identifiable up to exchange symmetry.
- Under mild regularity conditions (specifically regarding the support of covariates, e.g., containing an open subset), the likelihood is uniquely maximized on the quotient space $(\mathbb{R}^d \times \mathbb{R}^d) / \sim$ , where $(\beta, \gamma) \sim (\gamma, \beta)$ .
- This implies that while we cannot distinguish which vector is $\beta$ and which is $\gamma$ without external information, the set of parameters is uniquely determined.
Computational Approach:
- To handle the posterior bimodality caused by the symmetry, the authors employ a Pólya-Gamma Gibbs Sampler integrated with Replica Exchange (Parallel Tempering). This allows the Markov Chain Monte Carlo (MCMC) algorithm to jump between the two symmetric modes, ensuring efficient exploration of the parameter space.

C. Practical Solution: Relabeling Rule

Since applications often require a single ordered interpretation (e.g., distinguishing between "event occurrence" and "misclassification"), the authors propose a simple relabeling rule:

Fit a standard logistic regression (ignoring zero-inflation) to obtain $\hat{\beta}_{LR}$ .
Fit the ZILR model to obtain an ordered solution $(\hat{\beta}, \hat{\gamma})$ .
Compare the distance between $\hat{\beta}$ and $\hat{\beta}_{LR}$ against the distance between $\hat{\gamma}$ and $\hat{\beta}_{LR}$ .
Select the ordering where the first component is closer to $\hat{\beta}_{LR}$ .
Note: The authors acknowledge that $\hat{\beta}_{LR}$ is biased, but it serves as a heuristic anchor for ordering when no external prior knowledge exists.

3. Key Results

Simulation Studies

Bimodality Confirmation: In scenarios with continuous or mixed covariates, the posterior distribution clearly exhibited bimodality with two well-separated clusters corresponding to $(\beta, \gamma)$ and $(\gamma, \beta)$ . In purely binary designs, this separation was less distinct, correlating with theoretical conditions for identifiability.
Performance of Relabeling: The proposed relabeling rule successfully selected the correct parameter ordering in over 99% of replications (except in very low mislabel scenarios where the structural zero component was weakly identified).
Bias Reduction: The proposed ZILR approach with relabeling significantly reduced bias compared to:
- Standard Logistic Regression (which showed systematic negative shifts and sign flips).
- Naive ZILR (which retained random ordering, leading to high variance and inconsistent estimates).

Application to NHANES Data

The model was applied to self-reported diabetes data from the NHANES 2017–2018 survey.
The outcome was self-reported diabetes status, with covariates including age, BMI, insurance, and sex.
The analysis revealed two symmetric solutions with identical likelihoods. The relabeling rule selected the solution where the coefficients for the "event occurrence" component aligned more closely with the standard logistic regression estimates, providing a coherent interpretation of the data.

4. Significance and Implications

Theoretical Rigor: The paper resolves a long-standing ambiguity regarding the identifiability of ZILR models with shared designs. It clarifies that non-identifiability is not a total failure but a structured symmetry that can be mathematically characterized via quotient spaces.
Practical Guidance: It provides a concrete warning (the sign-flip theorem) against using standard logistic regression for zero-inflated data, which can lead to qualitatively wrong conclusions (e.g., concluding a risk factor is protective when it is actually a risk factor).
Computational Tools: The integration of Pólya-Gamma augmentation with Replica Exchange offers a robust computational framework for fitting these complex models, overcoming the trapping issues common in symmetric likelihood landscapes.
Heuristic for Interpretation: The proposed relabeling rule offers a practical, reproducible method for researchers to interpret results when external information distinguishing the two components is unavailable, bridging the gap between theoretical identifiability and applied inference.

In summary, this work provides the necessary theoretical foundation and practical tools to safely and effectively apply Zero-Inflated Logistic Regression in settings where covariate selection for the two components is uncertain or identical.