🤖 AI

Representation-driven Endoscopic Visual Embedding Alignment for Latent Generation

The paper introduces REVEAL, a large-scale generative foundation model for endoscopy trained on 5 million clinical frames that leverages domain-specific representation alignment to produce high-fidelity images and robust features for diverse clinical tasks, surpassing existing specialized models in performance and efficiency.

Original authors: Francisco Caetano, Tim J. M. Jaspers, Haiko Middeljans, Martijn R. Jong, Rixta A. H. van Eijck van Heslinga, Floor Slooter, Albert J. de Groof, Jacques J. Bergman, Peter H. N. De With, Fons van der So

Published 2026-08-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Francisco Caetano, Tim J. M. Jaspers, Haiko Middeljans, Martijn R. Jong, Rixta A. H. van Eijck van Heslinga, Floor Slooter, Albert J. de Groof, Jacques J. Bergman, Peter H. N. De With, Fons van der Sommen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers can learn to see and understand the human body just by looking at millions of pictures, much like a child learns what a cat is by seeing many different cats. This is the realm of computer vision, a branch of artificial intelligence where machines are taught to interpret visual data. A key tool in this field is the generative model, which is like a digital artist that doesn't just copy pictures but learns the "rules" of how things look so it can create brand-new, realistic images from scratch. Another crucial concept is representation alignment, which is essentially a teaching method where a student (the AI) tries to match its internal understanding of an image with a teacher's (a pre-trained expert AI). This paper tackles a specific, tricky corner of this science: endoscopy. This is the medical practice of using tiny cameras on flexible tubes to look inside the stomach and intestines. Why does this matter? Because doctors need huge amounts of diverse, high-quality images to train AI to spot diseases early, but patient privacy laws make sharing real medical photos incredibly difficult. If we can teach AI to generate perfect, fake medical images that look exactly like the real thing, we can solve the data shortage without ever compromising a patient's privacy.

Enter REVEAL, a new, super-powered AI artist designed specifically to paint pictures of the inside of the human gut. The researchers behind this project noticed a problem: most AI artists are trained on photos of everyday things like cats, cars, and landscapes. If you ask such an artist to draw a stomach lining, it might get the general shape right but miss the tiny, crucial details like the specific texture of the tissue or the way light reflects off the wet surface. These details are vital for doctors. To fix this, the team built REVEAL using a massive library of 5 million real endoscopic images collected from eight different hospitals in the Netherlands. Instead of teaching the AI with a "general knowledge" teacher, they trained a special "teacher" AI directly on these 5 million medical images first. Then, they used this expert teacher to guide REVEAL, ensuring that every new image the AI creates captures the intricate, bumpy, and shiny reality of the human digestive tract.

The results are quite impressive. REVEAL doesn't just make pretty pictures; it creates high-fidelity images that preserve the complex structures of the gut, something previous AI models struggled to do. But the paper suggests something even more surprising: this image-generator is also a brilliant detective. Even though it was never explicitly taught to diagnose diseases, the "brain" it uses to create images is so good at understanding the visual patterns of the gut that it can also be used to classify real medical images. In tests, REVEAL performed better than other specialized medical AI models, even when the images were blurry, too bright, or had other realistic distortions. The authors found that by sticking to medical data and using their specialized "teacher," they could build a model that is both a master artist and a sharp observer.

The paper explicitly argues against the idea that using general-purpose AI teachers (trained on regular photos) is good enough for medical tasks. They show that relying on these "out-of-domain" teachers leads to images that miss the subtle, clinical nuances of the human body. Instead, they suggest that for medical AI to truly work, it must be grounded in the specific data it will eventually handle. While the paper proves that REVEAL works well for generating images and extracting features, it stops short of claiming it is ready to diagnose patients in a hospital tomorrow. Instead, the authors propose that this model serves as a powerful foundation—a versatile starting point that other researchers can use to build tools for spotting rare diseases, filling in missing parts of images (like a digital version of "inpainting"), or detecting unusual patterns that don't fit the norm. By releasing their code and weights to the public, the team hopes to lower the barrier for others to build these life-saving tools, turning a massive, complex problem into a shared, solvable puzzle.

Technical Summary: REVEAL – Representation-driven Endoscopic Visual Embedding Alignment for Latent Generation

Problem Statement

The development of foundation generative models for endoscopy faces two primary hurdles: the significant domain gap between natural images and clinical endoscopic data, and the high computational cost associated with training large Diffusion Transformers (DiTs). While representation alignment has improved efficiency in general computer vision, its efficacy within the highly specialized endoscopic image space remains unproven. Existing strategies often rely on general-purpose vision encoders (e.g., ImageNet-pretrained models) that prioritize global semantic richness. However, endoscopic scenes are defined by subtle mucosal textures, specular reflections, and intricate anatomical morphologies where global semantic labels provide insufficient signal. Relying on out-of-domain semantic proxies risks suboptimal morphological fidelity and a failure to capture the nuanced clinical characteristics of the endoscopic manifold. Furthermore, existing endoscopic foundation models are predominantly discriminative; no prior work has explored large-scale generative pretraining as a pathway to clinical visual representations.

Methodology

The authors introduce REVEAL (Representation-driven Endoscopic Visual Embedding Alignment), a foundation generative model trained on GastroNet-5M (GN-5M), a multicenter dataset comprising 5 million unlabeled endoscopic frames.

Architecture and Training

REVEAL operates as a latent generative model within a compressed latent space, utilizing a Scalable Interpolant Transformer (SiT) as its backbone. The framework employs a dual-objective training strategy:

Denoising Objective: The model learns to reverse a stochastic interpolation process (Flow Matching) to synthesize images from noise.
Representation Alignment (REPA): To inject domain-specific knowledge, the hidden states of the student SiT model are aligned with a frozen, domain-adapted "teacher" vision encoder. This is achieved by maximizing the cosine similarity between the teacher's patch-level features and the student's hidden states.

Key Architectural Refinements

Building on the iREPA framework, REVEAL introduces specific modifications to preserve high-frequency anatomical details:

Convolutional Projection: Instead of a standard Multilayer Perceptron (MLP), a lightweight convolutional layer is used for the projection head. This introduces a local inductive bias, preserving spatial relationships between adjacent patch tokens and preventing the loss of fine textures.
Spatial Normalization: A spatial normalization layer is applied to the target representations to suppress global variance and enhance the signal-to-noise ratio of local anatomical structures, ensuring the model focuses on distinct structural signals.

The total loss function combines the denoising loss and the alignment loss ( $\mathcal{L} = \mathcal{L}_{\text{Denoising}} + \lambda \cdot \mathcal{L}_{\text{REPA}}$ ), with $\lambda$ set to 1.

Domain Adaptation

Crucially, the teacher encoder is not a general-purpose model but is pretrained directly on the GN-5M endoscopic distribution. The authors trained a DINOv3 ViT-B/16 on GN-5M using a two-stage curriculum to ensure the target representations are inherently aligned with the unique visual characteristics of endoscopic data.

Key Contributions

Systematic Benchmark of Alignment Design: The paper provides an extensive ablation study evaluating the influence of latent space choices (VAE backends), target representations, and model scaling (SiT-S to SiT-L) on both morphological fidelity and convergence.
Robustness and Generalization Evaluation: The model is evaluated across multiple independent datasets, including a corrupted robustness benchmark (BE-C) featuring realistic imaging artifacts (motion blur, defocus, overexposure, etc.), demonstrating generalization beyond the training distribution.
Largest Endoscopic Generative Foundation Model: REVEAL represents the largest generative foundation model for endoscopy to date, trained on 5 million frames. It serves a dual purpose: a high-fidelity image generator and a competitive feature extractor.

Results

Generative Performance

Fidelity: The optimal configuration uses the SD2 latent space and the DINOv3-B (GN-5M) teacher. Scaling the model to SiT-L/2 and training on the full GN-5M dataset achieved a Fréchet Inception Distance (FID) of 5.32, significantly outperforming models trained on subsets or with general-purpose teachers.
Qualitative Synthesis: REVEAL produces high-fidelity images that capture realistic mucosal textures and specular highlights. It maintains structural coherence in spatial editing tasks such as inpainting and outpainting, respecting the geometric constraints of the gastrointestinal manifold.

Discriminative Performance (Feature Extraction)

Despite being a generative model not explicitly optimized for classification, REVEAL's frozen SiT blocks serve as a powerful feature extractor:

Benchmarking: On the POLAR (polyp classification) and BE (Barrett's Esophagus) benchmarks, REVEAL achieved competitive results, surpassing established endoscopic foundation models like EndoViT and Endo-FM.
Robustness: Under corrupted imaging conditions (BE-C), REVEAL demonstrated superior robustness compared to EndoViT, which degraded catastrophically. REVEAL maintained competitive performance, outperforming general-purpose encoders (SAM2, DINOv2, DINOv3) and domain-specific discriminative models.

Significance and Claims

The paper posits that representation alignment with domain-adapted visual priors is the primary driver of synthesis fidelity in medical imaging. The authors claim that:

Domain Specificity is Critical: In-domain encoders consistently outperform general-purpose alternatives, proving that reliance on out-of-domain semantic proxies is insufficient for capturing clinical nuances.
Generative Pretraining as a Viable Path: Large-scale generative pretraining is a viable and complementary path to masked image modeling for building clinical visual representations.
Dual Utility: The aligned latent space offers a versatile foundation for downstream tasks, including conditional synthesis for rare pathology generation, segmentation via generative features, and out-of-distribution detection.
Lowering the Threshold: By providing a high-capacity backbone trained on massive clinical data, REVEAL lowers the computational threshold for developing specialized clinical tools, offering an open, versatile foundation for future intelligent gastroenterology systems.

The authors conclude that while the model is not explicitly trained for discriminative tasks, its ability to surpass specialized discriminative models under both clean and corrupted conditions highlights the value of learning robust, anatomically coherent representations through generative pretraining.