📊 statistics

Decoupling risk and masking in mammographic density under irregular follow up using a latent Markov progression detection framework

This paper proposes a two-phase latent Markov framework that decouples breast cancer risk from mammographic masking by modeling an underlying latent disease progression separate from observed density, thereby quantifying how irregular screening and masking effects can significantly underestimate risk in specific patient groups.

Original authors: Furkan Danisman, Zarina Oflaz, Zeynep Kalaylioglu, Mahmut Onur Kulturoglu, Lutfi Dogan

Published 2026-07-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Furkan Danisman, Zarina Oflaz, Zeynep Kalaylioglu, Mahmut Onur Kulturoglu, Lutfi Dogan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a mystery inside a foggy room. You have a flashlight (the mammogram) that helps you see, but the room is filled with thick, white fog (dense breast tissue). This fog makes it harder to spot the bad guys (cancer cells), a problem scientists call "masking." At the same time, having a lot of fog in the room is actually a sign that the room itself is more likely to have bad guys hiding in it. So, the fog is a double-edged sword: it hides the danger, but its presence also suggests the danger is there.

For a long time, doctors have looked at how much fog is in the room to guess how likely a patient is to get sick. But there's a tricky problem: the fog changes over time, and patients don't visit the doctor on the same schedule. Some come every year, some every two years, and some have gaps in between. It's like trying to track a character's journey in a video game when you only get to see them at random, irregular moments. If you try to guess the whole story from these scattered snapshots, you might get the plot wrong. This paper tackles that messy, irregular data to figure out the real story of risk, separating the "fog" that hides the cancer from the "danger" that causes it.

The Detective's New Map

The authors, a team of statisticians and doctors, decided to stop looking at the "fog" (mammographic density) as the direct cause of risk. Instead, they imagined a hidden, invisible engine driving the whole process. Think of this engine as a secret "progress bar" inside a patient's body that moves through different levels of danger. The fog we see on the X-ray is just a noisy, imperfect reflection of where that progress bar is sitting.

To solve the puzzle, they built a two-step detective framework.

Step 1: Cleaning Up the Messy Timeline
First, they had to deal with the irregular visits. Since patients showed up at random times, the data looked like a scattered trail of breadcrumbs. The team used a clever trick to "regularize" this data. They assumed that breast density usually follows a one-way street: it tends to get less dense over time (like fog clearing up), rarely going backward. Using this rule, they filled in the missing months between visits, creating a smooth, month-by-month timeline for every patient. It's like taking a blurry, stop-motion video and using the rules of physics to fill in the missing frames so you can see the smooth motion.

Step 2: The Hidden Engine and the Fog
Next, they used a "Latent Markov Model." In plain English, this is a way of guessing the invisible "progress bar" (the latent state) based on the visible fog. They assumed the body moves through four hidden levels of risk: Low, Emergent, Established, and Critical.

The Big Idea: They proposed that the risk comes from how much time you spend in these hidden levels, not from the fog itself. The fog is just a "detectability factor." If you are in a high-risk hidden state but have very dense fog, the cancer might be hiding. If you are in the same high-risk state but have less fog, the cancer is easier to spot.

By separating the "risk engine" from the "fog," they could finally ask: How much does the fog trick us into thinking we are safer than we actually are?

What They Found

The team looked at 616 patients from a hospital in Turkey. They split them into two groups based on Body Mass Index (BMI): "Overweight" and "Obese."

The Hidden States Make Sense
Their model successfully identified the four hidden risk levels. They found that patients generally moved from the "Critical" or "Established" risk states toward the "Low" or "Emergent" states over time. This is good news! It means that, on average, people in the study were moving toward safer zones. However, the model also showed that some patient characteristics made this journey harder:

Family History: Patients with a family history of breast cancer tended to get "stuck" in the higher-risk states longer.
Parity (Number of Births): Having more children was linked to moving faster toward lower-risk states.
Age and Menopause: The rules changed depending on whether a patient was overweight or obese. For example, in the overweight group, being post-menopausal usually meant starting in a safer state, but in the obese group, the pattern was different.

The "Masking" Shock
The most exciting part of the study was testing the "masking" effect. The researchers ran their numbers three times with different assumptions about how well the fog hides the cancer:

Scenario A (No Masking): They pretended the fog never hid anything (like looking through a clean window).
Scenario B (Moderate Masking): They assumed the fog made it harder to see, but not impossible.
Scenario C (Heavy Masking): They used realistic numbers where dense fog makes it very hard to spot cancer.

The Result: When they switched from the "No Masking" view to the "Realistic Masking" view, the estimated risk jumped significantly.

For post-menopausal overweight patients, the estimated risk went up by about 70%.
For obese patients (regardless of menopause), the risk went up by about 40%.

This means that if we ignore the fact that dense tissue hides cancer, we are seriously underestimating how dangerous the situation is for these specific groups. The paper suggests that for these patients, the "fog" is doing a better job of hiding the danger than we thought, and we might need to be more aggressive with screening to catch it.

The Takeaway

This paper doesn't claim to have cured cancer or invented a new machine. Instead, it built a better map. It showed that by separating the "hidden risk" from the "visible fog," we can see the true danger more clearly. The study suggests that for women with higher BMI, the current way we calculate risk might be too optimistic because it doesn't fully account for how much the dense tissue is hiding the cancer.

The authors are careful to say this is based on their specific data and simulations, but the message is clear: the fog matters. It's not just a sign of risk; it's a shield for the risk. By understanding this shield, doctors might be able to spot the bad guys earlier, especially in the groups where the fog is thickest.

Technical Summary: Decoupling Risk and Masking in Mammographic Density

Problem Statement
Mammographic breast density is a well-established independent risk marker for breast cancer but simultaneously induces a "masking effect" that reduces mammographic sensitivity, particularly in dense breasts. This dual role complicates risk inference: high density correlates with elevated underlying risk but also obscures existing tumors. Current literature often struggles to disentangle the risk associated with the underlying disease process from the detectability issues caused by density. Furthermore, screening histories are typically longitudinal and observed at irregular intervals, a feature often ignored or inadequately handled by standard time-to-event or joint models, potentially biasing risk estimates. Additionally, many existing machine learning approaches prioritize predictive accuracy over interpretability and fail to account for the longitudinal structure of disease progression.

Methodology
The authors propose a two-phase framework designed to handle irregular follow-up and decouple cancer risk from detection probability. The methodology is stratified by Body Mass Index (BMI) groups (specifically overweight and obese in this study) to account for differences in how density categories map to risk profiles.

Data Regularization (Pre-processing):
To address irregular visit times, the authors discretize the follow-up window (2017–2024) into monthly intervals. They exploit the near-monotonic nature of mammographic density decline (due to breast involution) to impute missing values. Missing gaps between observed visits are filled using a monotone Markov chain transition model, ensuring that imputed paths respect the observed endpoints and the biological constraint that density generally moves from higher (D) to lower (B) categories.
Phase 1: Latent Markov Model (LMM) for Progression:
The core assumption is that mammographic density is an observed manifestation of an unobserved latent disease process. The framework models the regularized density sequence as arising from a latent Markov process with $K=4$ states (Low Risk, Emergent Risk, Established Risk, Critical Risk).
- Structure: The model includes an initial state distribution, transition probabilities between latent states, and emission probabilities linking latent states to observed density categories.
- Goal: This phase extracts the "risk-relevant" information (the latent trajectory) from the "detectability-relevant" information (the observed density). The model is fitted separately for overweight and obese groups using the Expectation-Maximization (EM) algorithm.
Phase 2: Decoupling Detection and Risk:
This phase links the inferred latent history to cancer diagnosis while explicitly modeling the masking effect.
- Assumption 1: Conditional on the cumulative time spent in each latent state, the observed density trajectory carries no additional information about cancer development.
- Assumption 2: Conditional on cancer being present, detection depends on the density trajectory only through the final observed density category (acting as a sensitivity parameter).
- Model: A detection-risk model is formulated where the probability of a positive diagnosis is the product of the probability of cancer development (modeled via logistic regression on latent state occupancy counts and covariates) and the detection probability ( $p_c$ ) associated with the final density category.
- Sensitivity Analysis: Since detection probabilities ( $p_B, p_C, p_D$ ) are not identifiable from the data alone, they are treated as sensitivity parameters. The authors evaluate risk estimates under three scenarios: (i) no masking ( $p=1$ ), and two empirically supported masking scenarios where detection probability decreases with increasing density ( $p_D < p_C < p_B$ ).
Uncertainty Quantification:
The framework propagates uncertainty from three sources: (i) stochastic completion of density histories, (ii) uncertainty in the inferred latent paths, and (iii) parameter estimation uncertainty. This is achieved through a nested resampling procedure involving random density completions, posterior path sampling (Monte Carlo), and subject-level bootstrapping.

Key Results

Latent State Structure: The LMM identified four distinct latent states. The mapping of observed density to these states differed by BMI; for instance, in the overweight group, the highest density category (D) split into two distinct latent risk states, whereas in the obese group, the intermediate density category (C) showed this heterogeneity.
Progression Patterns: Patients generally moved from higher-risk to lower-risk latent states over time.
- Covariate Effects: Higher parity was associated with faster movement toward lower-risk states. A family history of breast cancer was associated with longer persistence in higher-risk states.
- Menopausal Status: The effect of menopausal status on progression varied by BMI. In the overweight group, pre-menopausal status facilitated de-escalation to lower risk, whereas in the obese group, the pattern was reversed.
Impact of Masking on Risk Estimation:
- When moving from a "no-masking" assumption to empirically supported masking scenarios, the estimated odds of breast cancer diagnosis increased significantly.
- For post-menopausal overweight patients, the estimated risk increased by approximately 70–75%.
- For obese patients (regardless of menopausal status), the estimated risk increased by approximately 40%.
- The latent state occupancy (time spent in riskier states) remained a significant predictor of diagnosis across all scenarios, confirming the validity of the latent progression assumption.

Significance and Claims
The paper claims to provide a methodological framework that successfully decouples risk from detectability. By attributing risk to a latent disease process and treating observed density primarily as a factor of detectability, the authors demonstrate that standard analyses ignoring masking substantially underestimate breast cancer risk, particularly in subgroups with high density and specific BMI/menopausal profiles.

The authors argue that this underestimation is large enough to justify clinical reconsideration of early intervention strategies and the potential need for multimodal surveillance (e.g., supplemental imaging) for high-risk subgroups currently masked by density. The framework offers an interpretable alternative to "black-box" predictive models, explicitly modeling how risk accumulates over time and how masking distorts the observed relationship between density and diagnosis. The study concludes that accounting for the longitudinal structure of density and the mechanics of masking is essential for accurate risk inference in irregular screening data.

The Detective's New Map

What They Found

The Takeaway

Technical Summary: Decoupling Risk and Masking in Mammographic Density

More like this