🤖 AI

Longitudinal Multi-View Breast Cancer Risk Prediction

This paper introduces LMV-Net, a deep learning model that improves breast cancer risk prediction by jointly analyzing anatomically complementary CC and MLO views within an explicitly aligned longitudinal framework, outperforming existing methods across various patient subgroups.

Original authors: Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstrøm, Elisabeth Wetzer, Stine Hansen, Robert Jenssen, Michael Kampffmeyer

Published 2026-07-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstrøm, Elisabeth Wetzer, Stine Hansen, Robert Jenssen, Michael Kampffmeyer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict if a hidden treasure (cancer) is about to appear in a complex, foggy landscape (a breast). For years, doctors have looked at this landscape using two different camera angles: a top-down view (CC) and a side-angled view (MLO). They also check old photos to see how the landscape has changed over time. But here's the problem: most computer programs trying to help doctors were looking at these clues in isolation. Some only looked at one camera angle, while others compared old and new photos without actually lining them up perfectly, like trying to compare two maps of the same city that were drawn at different scales and slightly shifted.

The authors of this paper, Solveig Thrun and her team, built a new tool called LMV-Net to fix this mess. Think of LMV-Net as a super-smart detective who doesn't just look at the photos; they actively stretch and warp the old photos to fit perfectly over the new ones, pixel by pixel. This "explicit alignment" is crucial because it lets the computer spot tiny, subtle changes in the tissue that would otherwise be lost in the shuffle.

But the detective doesn't stop there. They use a special "Dual Stream Attention" mechanism. Imagine the detective has two pairs of glasses: one for the top-down view and one for the side view. Instead of looking through them separately, the glasses talk to each other. If the top-down view sees something suspicious, it tells the side-view glasses, "Hey, look right there!" This allows the system to combine the best parts of both angles to get a complete picture of the tissue.

What the paper says works (and what it rules out):
The researchers tested this new detective on two huge sets of real-world mammogram data (called EMBED and CSAW-CC). They found that LMV-Net consistently outperformed the previous best methods. In fact, on the EMBED dataset, it improved the ability to predict risk over five years by about 6.7% compared to the runner-up.

The paper explicitly argues against the idea that you can get the best results by just looking at a single camera angle or by comparing old and new images without carefully lining them up first. Their tests showed that models which ignored the second view or didn't align the images properly performed significantly worse. For example, if you removed the "Dual Stream Attention" (the talking glasses), the performance dropped noticeably. If you only looked at one view, the scores fell even harder.

How sure are they?
The authors are quite confident in these results because they didn't just guess; they measured it. They ran the model on thousands of patients and calculated a "C-index" (a score of how good the prediction is) and an "AUC" (another score of accuracy).

On the EMBED dataset, their best model (with a fine-tuned "brain") reached a C-index of 81.4% for 1-year predictions and 80.0% for 5-year predictions.
On the CSAW-CC dataset, it reached 74.4% for 1-year and 73.5% for 5-year predictions.
These numbers were statistically significant, meaning the improvement wasn't just luck. The paper notes that the gains were even bigger for women with dense breast tissue, which is usually the hardest group to screen.

The "Magic" of the Attention Maps
To prove their detective was actually looking at the right things, the authors looked at the computer's "attention maps"—basically, a heat map showing where the AI was looking. They found that the model focused 8 to 10 times more on the actual tumor area than on the surrounding healthy tissue. This suggests the AI isn't just guessing; it's zeroing in on the suspicious spots, just like a radiologist would.

What's next?
The paper suggests that this approach is a strong step forward for personalized screening, potentially helping doctors identify high-risk patients earlier. However, the authors are careful to say this isn't a "solved problem" yet. They note that future work could involve looking at more than just two time points (maybe three or four old photos instead of one) to get an even richer history. They also mention that while their method works well, it relies on the ability to align images perfectly, so better alignment tools could make it even stronger.

In short, LMV-Net suggests that by carefully lining up old and new photos from two different angles and letting those angles "talk" to each other, we can spot breast cancer risks more accurately than ever before, especially for those with dense tissue. It's a promising new tool, but the authors remind us that the journey to perfect personalized screening is still ongoing.

Technical Summary: Longitudinal Multi-View Modeling for Breast Cancer Risk Prediction (LMV-Net)

Problem Statement
Accurate breast cancer risk prediction from screening mammography is essential for enabling personalized screening intervals and early detection. While recent deep learning methods have demonstrated the value of longitudinal data (comparing current and prior screenings) and explicit temporal alignment, existing approaches suffer from significant limitations. Current models either perform explicit alignment using only a single mammographic view (e.g., CC or MLO) or model multiple views without explicit longitudinal alignment. Consequently, these methods fail to fully exploit the complementary spatial–temporal information utilized in clinical practice, where radiologists compare both Craniocaudal (CC) and Mediolateral Oblique (MLO) views across time to assess subtle tissue changes.

Methodology: LMV-Net
The authors propose LMV-Net, a breast-wise longitudinal multi-view risk prediction model designed to jointly analyze anatomically complementary CC and MLO views within an explicitly aligned longitudinal framework. The architecture consists of six sequential stages:

Feature Extraction: A shared image encoder (based on a ResNet-18 backbone, specifically the Mirai encoder) extracts feature maps from current and prior mammograms for both CC and MLO views.
Temporal Feature Alignment: To address non-rigid breast deformation, the model explicitly aligns longitudinal feature maps in feature space. It utilizes a pre-trained, frozen MammoRegNet to generate image-based deformation fields that register prior mammograms to current ones. Separate deformation fields are estimated for each view (CC and MLO), and a spatial transformer function aligns the prior feature maps to the current ones.
Temporal Subtraction: View-specific difference features are computed by subtracting the aligned prior features from the current features ( $f_{diff} = f_{cur} - f_{pri-align}$ ). Positional encoding is applied to these difference features to account for variable time intervals between screenings.
Temporal Feature Concatenation: Current features, aligned prior features, and difference features are concatenated along the channel dimension to form longitudinal feature representations ( $f_{long}$ ) for each view, integrating appearance, context, and temporal change.
Dual Stream Attention: A novel Dual Stream Attention block refines the longitudinal features. This module employs parallel mechanisms:
- Self-Attention: Models long-range intra-view interactions to improve spatial coherence within the same view.
- Cross-Attention: Leverages features from the complementary view (e.g., MLO features as keys/values for the CC stream) to enable selective attention to anatomically relevant regions across views.
- The outputs are fused via element-wise summation, followed by Layer Normalization, DropPath (stochastic depth), and a Feed-Forward Network (FFN) with residual connections.
Final Fusion and Risk Prediction: The refined features from both views are pooled and concatenated into a fused multi-view representation. This passes through a cumulative probability layer comprising fully connected layers to predict hazards at future time steps (1–5 years). The model is trained using a time-dependent binary cross-entropy loss with masking for censored patients, employing multilevel joint learning to capture dependencies between views.

Key Contributions

LMV-Net Architecture: A novel framework that facilitates multi-view, multi-time-point integration via explicit feature-space alignment and cross-view temporal fusion, closely mirroring radiologist workflows.
Dual Stream Attention Block: A mechanism that refines longitudinal features of each view using parallel self- and cross-attention, effectively capturing both intra-view dependencies and inter-view complementary information.
Extensive Validation: Robust evaluation on two large public datasets (EMBED and CSAW-CC), demonstrating consistent performance improvements over state-of-the-art methods.

Results
The model was evaluated on the EMBED and CSAW-CC datasets, comparing performance against state-of-the-art baselines including Mirai, VMRA-MaR, OA-BreaCR, and ImgFeatAlign.

Overall Performance: LMV-Net consistently outperformed all baselines in C-index and AUC across 1–5 year risk horizons on both datasets. On EMBED, the fine-tuned LMV-Net achieved a C-index of 81.4% (1-year) and 80.0% (5-year), representing a statistically significant improvement ( $p < 0.05$ ) of approximately 6.7% over the best baseline. On CSAW-CC, it achieved a C-index of 74.4% (1-year) and 73.5% (5-year), with a 1.7% improvement over the best baseline.
Subgroup Analysis: The model demonstrated particular robustness in challenging scenarios, achieving the highest C-index across nearly all breast density categories (low, medium, high) and cancer types (invasive vs. others). Improvements were most pronounced in medium- and high-density groups.
Ablation Studies: Removing key components (longitudinal alignment, multi-view modeling, dual-stream attention, or encoder fine-tuning) consistently degraded performance, confirming the necessity of each architectural element.
Attention Analysis: Visualization of attention maps revealed that the Dual Stream Attention block concentrates significantly higher attention density (8–10×) on tumor regions compared to surrounding tissue, validating its ability to focus on clinically relevant areas.

Significance and Claims
The paper claims that LMV-Net addresses a critical gap in breast cancer risk prediction by successfully integrating explicit longitudinal alignment with multi-view analysis. The authors state that their approach:

Enhances Risk Stratification: By leveraging complementary spatial-temporal cues, the model provides more reliable risk assessments, particularly for women with dense breast tissue where traditional sensitivity is reduced.
Mimics Clinical Workflow: The explicit alignment and dual-view processing reflect how radiologists compare prior and current exams, leading to more clinically relevant feature extraction.
Enables Personalized Screening: The improved accuracy in identifying high-risk patients supports the development of personalized screening strategies and more efficient allocation of screening resources.

The authors conclude that while their results demonstrate the value of joint longitudinal multi-view modeling, future work should explore incorporating multiple prior examinations for richer temporal context and evaluating additional metrics like Precision-Recall AUC to address class imbalance.

Technical Summary: Longitudinal Multi-View Modeling for Breast Cancer Risk Prediction (LMV-Net)

More like this