Training-inference input alignment outweighs framework choice in longitudinal retinal image prediction

1. Problem Statement

Progressive retinal diseases, such as Geographic Atrophy (GA) and Stargardt macular dystrophy, cause irreversible vision loss through the expansion of hypoautofluorescent lesions. Current clinical management relies on qualitative visual comparisons of serial images or scalar progression scores, which discard valuable spatial and intensity information.

While recent deep learning approaches have moved toward increasingly complex generative models (e.g., neural ODEs, conditional diffusion, latent diffusion pipelines) to predict future retinal images, it remains unclear whether this complexity is necessary for slowly progressing diseases. A critical, often overlooked issue in conditional generation is the training-inference distributional mismatch: models are typically trained to denoise corrupted versions of the target image (which is unavailable at inference), while inference is initialized from pure noise or a different input distribution. This paper investigates whether correcting this mismatch is more impactful than increasing generative complexity.

2. Methodology

A. Controlled Comparison of Conditioning Configurations

The authors conducted a controlled experiment using a single architecture (U-Net) and dataset, varying only the conditioning configuration to isolate the effects of distributional alignment versus generative framework. Five configurations were tested:

1. Std-DDIM-50step: Standard diffusion training (noising the target $I^*$) with 50-step inference from pure Gaussian noise (Maximum Mismatch).
2. Std-DDIM-1step: Standard training (noising $I^*$) but single-step inference initialized from the noised last observation ($I_N$) (Partial Mismatch).
3. IA-Nonlinear: Training on noised $I_N$ (last observation) with single-step inference (Full Alignment, Stochastic).
4. IA-Linear: Stochastic training with linear interpolation noise schedule on $I_N$ (Full Alignment, Stochastic).
5. TRU (Temporal Retinal U-Net): Deterministic direct regression on $I_N$ with no noise injection (Full Alignment, Deterministic).

B. The Proposed Model: TRU

Based on the findings, the authors developed TRU, a deterministic direct-regression model designed to match the low-entropy nature of the task.

• Architecture: A multiscale temporal U-Net (~22.8M parameters).
• Input: The most recent observation ($I_N$), full longitudinal history ($\mathcal{H}$), and continuous time-delta conditioning.
• Key Mechanisms:
  • Continuous Time-Delta Conditioning: Encodes the time difference between history frames and the prediction target using log-sinusoidal embeddings.
  • Multi-Scale History Aggregation: Uses delta-weighted attention to aggregate features from previous visits, prioritizing temporally proximal frames.
  • Residual Prediction: Predicts the change (residual) from the last observation rather than the absolute target image, concentrating capacity on disease-relevant signals.
  • Deterministic Inference: Single forward pass with no stochastic sampling.

C. Evaluation Framework

The model was evaluated on 28,902 eyes across four cohorts:

1. Optos FAF Mixed-Disease: Primary benchmark (9,942 eyes).
2. Optos Stargardt: Zero-shot transfer to a rare inherited disease (288 eyes).
3. Heidelberg Spectralis Stargardt: Zero-shot cross-vendor transfer (125 eyes).
4. Glaucoma en-face SLO: Zero-shot cross-modality transfer (18,547 eyes).

Metrics: Primary metrics focused on progression dynamics ($\Delta$SSIM of change maps) and lesion-level accuracy (Dice, HD95), supplemented by pixel-level fidelity (SSIM, PSNR, MAE).

3. Key Findings & Results

A. Distributional Alignment is the Dominant Factor

The controlled comparison revealed that distributional alignment between training and inference inputs is the primary driver of performance, far outweighing the choice of generative framework.

• Impact of Mismatch: Moving from maximum mismatch (Std-DDIM-50step) to partial alignment (Std-DDIM-1step) improved $\Delta$SSIM by +0.046 and SSIM by +0.079.
• Impact of Full Alignment: Further aligning the training input (noising $I_N$ instead of $I^*$) added another +0.036 to $\Delta$SSIM.
• Framework Equivalence: Once alignment was achieved, the choice between stochastic frameworks (IA-Nonlinear, IA-Linear) and deterministic regression (TRU) produced no statistically significant difference on primary clinical metrics ($p > 0.05$).

B. Mechanistic Explanation: Low Task Entropy

The authors provided a mechanistic explanation for why stochastic sampling offers no benefit in this domain:

1. Task Entropy Analysis: Inter-visit changes in retinal images are dominated by time-invariant acquisition variability (illumination, alignment) rather than disease progression. The correlation between elapsed time and pixel change is weak ($r=0.116$).
2. Posterior Concentration: Analysis of stochastic models showed that the conditional posterior $p(I^* | I_N, \mathcal{H})$ collapses to an effective point mass. Across 10 independent noise realizations, the variance contributed less than 0.02% to the total prediction error; 99.99% of the error was systematic bias.
3. Conclusion: The predictable disease signal occupies a low-dimensional subspace. Stochastic sampling only generates diversity in the clinically uninformative acquisition noise dimensions.

C. Performance of TRU

• Benchmarking: TRU outperformed state-of-the-art baselines (ImageFlowNet, TADM, 2D-BrLP) on all primary metrics across all cohorts.
• Zero-Shot Generalization: TRU achieved strong performance on unseen Stargardt disease and cross-vendor Spectralis data without retraining.
• Cross-Modality: TRU generalized to Glaucoma SLO images (different modality and disease), outperforming all other methods, though absolute performance was lower than on FAF, delineating the boundary of modality shift.
• History Length: TRU's advantage grew monotonically with the number of available history frames, making it particularly valuable for patients with long-term monitoring data.

4. Key Contributions

1. Isolation of Distributional Alignment: Demonstrated that correcting the training-inference mismatch yields gains an order of magnitude larger than switching generative frameworks, challenging the trend toward increasing model complexity.
2. Mechanistic Insight: Established that for slowly progressing retinal diseases, the conditional posterior collapses to a point mass due to low task entropy, rendering stochastic machinery unnecessary.
3. TRU Model: Introduced a simple, deterministic, history-conditioned model that matches or exceeds complex diffusion-based methods while offering reproducible inference and lower computational cost.
4. Diagnostic Principle: Proposed a task-adaptive design principle: practitioners should measure task entropy and posterior concentration before selecting a framework. If the posterior collapses, deterministic regression is sufficient.

5. Significance and Implications

• Paradigm Shift: The paper argues against the "bigger is better" trajectory in medical image generation. It suggests that for many longitudinal tasks, the bottleneck is not model capacity but data alignment.
• Clinical Utility: Deterministic inference (TRU) is crucial for clinical adoption, ensuring that the same patient history always yields the same prediction, which is essential for regulatory approval and clinician trust.
• Efficiency: By avoiding complex diffusion sampling (50+ steps) and massive latent pipelines, TRU offers a computationally efficient solution that scales better with available history.
• Generalizability: The proposed diagnostic framework (measuring task entropy and posterior concentration) can be applied to other longitudinal medical imaging domains (e.g., neurodegeneration, tumor monitoring) to determine if stochastic generative models are actually warranted.

In summary, the paper demonstrates that distributional alignment supersedes generative complexity. For slowly progressing retinal diseases, a simple, aligned, deterministic regression model (TRU) is superior to complex, misaligned, or stochastic generative models, offering a more robust, efficient, and clinically interpretable solution.