💻 computer science

Modern Computer Vision from CNNs to Foundation Models: A Unified Framework for Representation Learning

This survey presents a unified representation learning framework that traces the convergence of discriminative, multimodal, and generative paradigms in modern computer vision—from CNNs and Vision Transformers to foundation and diffusion models—arguing for a cohesive perspective to advance general-purpose visual intelligence.

Original authors: Abdelmalik Moujahid, Fadi Dornaika, Salah-Eddine Mechkouri, Jesús Cigales Canga

Published 2026-08-04

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Abdelmalik Moujahid, Fadi Dornaika, Salah-Eddine Mechkouri, Jesús Cigales Canga

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to see the world. For a long time, the best way to do this was to give the robot a very specific, rigid set of rules, like a chef following a recipe step-by-step. But the world is messy, colorful, and full of surprises. Recently, scientists have been trying to build "foundation models"—massive, super-smart brains that don't just follow a recipe but actually learn what things look like by studying millions of pictures, just like a child learns by looking around. These models are now so powerful that they can not only identify a cat in a photo but also describe it, draw a new one from scratch, or even understand how it sounds. The big question researchers are asking is: Are all these different types of smart robots actually just different versions of the same thing? Are they all trying to build the same kind of internal "map" of the world, just using different tools to get there?

This paper, written by a team of researchers, acts like a grand tour guide for this rapidly changing landscape of computer vision. Instead of treating different types of AI as separate tribes, the authors propose a "unified framework." They suggest that whether a model uses old-school convolutional layers (which look at small patches of an image), modern "Transformers" (which look at the whole picture at once), or generative models (which create new images), they are all fundamentally doing the same job: constructing and manipulating a hidden "latent representation" of the world. Think of this representation as a secret, compressed language the AI uses to understand reality. The paper argues that the boundaries between models that recognize things, models that talk about them, and models that draw them are blurring. They are all converging toward a single, more general-purpose intelligence that can handle perception, reasoning, and creation all at once.

The Evolution of the Robot's Eyes

To understand where we are, we have to look at how the robot's "eyes" have changed. For over a decade, the dominant tool was the Convolutional Neural Network (CNN). You can think of a CNN like a tiny, local detective. It has a small magnifying glass (a filter) that it slides across an image, looking for simple patterns like edges or corners. It's very good at this local work and is great at learning from small amounts of data because it has built-in rules (inductive biases) about how the world works, like "objects usually have parts close together." However, this detective has a blind spot: it struggles to connect the dots between things that are far apart in the image. It might see a dog's ear and a dog's tail but fail to realize they belong to the same dog if they are on opposite sides of the photo.

Then came the Vision Transformer (ViT). If the CNN is a local detective, the ViT is a global gossip. Instead of sliding a magnifying glass, it breaks the image into little puzzle pieces (patches) and lets every piece talk to every other piece at the same time. This is done through a mechanism called "self-attention." Suddenly, the robot can see the whole picture at once. It can instantly connect the ear and the tail, no matter how far apart they are. This made AI incredibly powerful, but it came with a cost: these "gossip" models are hungry. They need massive amounts of data to learn because they don't have the same built-in rules as the CNNs; they have to figure out the rules of the world from scratch.

The Great Convergence: Mixing the Tools

The paper highlights a fascinating trend: the two approaches are starting to merge. Researchers realized that the local detective and the global gossip each have strengths the other lacks. So, they started building Hybrid Architectures. Imagine a team where the local detective does the initial scanning to find the edges, and then the global gossip takes over to understand the whole scene. Models like CoAtNet and MaxViT do exactly this. They use convolutional layers to get the local details efficiently and then switch to attention mechanisms to understand the big picture. The paper suggests this isn't just a random mix; it's a necessary evolution to get the best of both worlds: the data efficiency of CNNs and the powerful reasoning of Transformers.

The Rise of the "Foundation" Brain

The next big leap described in the paper is the move toward Foundation Models. These are the "super-brains" of the AI world. Instead of training a robot to just recognize cats, scientists started training them on billions of images without any labels (self-supervised learning). Models like DINO and DINOv2 are examples. They learn to understand the structure of the world just by looking at pictures. If you show them a picture of a cat from the front and then from the side, they learn that these are the same object, even without anyone telling them "that's a cat."

The paper explains that these models are becoming so good at understanding visual representations that they can be used for almost anything. DINOv3, for instance, creates such detailed maps of an image that you can use them to find specific parts of an object or match similar objects across different photos, all without needing specific training for those tasks. It's like the robot has learned the "grammar" of vision, so it can speak any "dialect" (task) you ask it to.

Speaking and Drawing: Multimodal and Generative Models

The story gets even more exciting when these visual brains start talking and drawing. Multimodal models like CLIP and ImageBind connect the visual world to the world of words and sounds. CLIP learns to match images with text descriptions. It's like teaching the robot that the picture of a "sunset" and the words "sunset" mean the same thing. ImageBind takes this further, binding images to audio, depth, and even motion sensors, creating a single shared space where all these different senses live together.

Then there are the Generative Models, specifically Diffusion Models. These are the artists. They don't just recognize a picture; they create one. They start with a canvas full of random static (noise) and slowly, step-by-step, remove the noise to reveal a clear image. The paper notes that these models are now using the same "Transformer" brains that were used for recognition. Diffusion Transformers (DiT) replace the old convolutional artists with the new global gossip brains, making the art generation faster and more consistent. Even newer methods like Rectified Flow are trying to make this process even more direct, turning the slow, noisy journey into a straight, smooth line.

The Big Picture: One Unified Language

The core finding of this paper is that all these different approaches—CNNs, Transformers, self-supervised learning, multimodal systems, and generative art—are not separate islands. They are all converging on a single, unified way of thinking about vision. The authors argue that we should stop looking at them as different tools and start seeing them as different ways of building and using a latent representation.

Think of this latent representation as a secret, compressed language that the AI uses to describe the world.

CNNs build this language using local rules.
Transformers build it by letting everything talk to everything.
Foundation Models learn this language by reading the whole library of the internet.
Generative Models use this language to write new stories (create images).

The paper suggests that the future of computer vision isn't about picking one winner. Instead, it's about building General-Purpose Visual Intelligence. These future systems will be able to see, understand, talk, and create all within the same framework. They will be able to look at a photo, explain what's happening, answer questions about it, and even draw a new version of it, all because they share the same underlying "brain" for understanding the world.

The Road Ahead: Challenges and Dreams

While the paper is optimistic about this convergence, it also points out that we aren't there yet. There are still some big hurdles.

Efficiency: These massive models are hungry for computing power. Making them run on small devices (like phones or robots) is still a challenge.
Trust and Safety: Because these models are so complex, it's hard to know why they make certain decisions. If a medical AI says a patient has a disease, we need to know if it's right or if it's just guessing. The paper emphasizes the need for "trustworthy deployment," meaning models that are reliable and can explain themselves.
Data: These models need huge amounts of high-quality data. Finding enough good data without copying everything from the internet is a growing problem.

In conclusion, this paper paints a picture of a field that is maturing. We are moving away from building specialized robots for single tasks (like just counting cars) toward building universal visual assistants that can learn, reason, and create. The tools are changing, but the goal is becoming clearer: to create machines that see the world not just as a collection of pixels, but as a rich, interconnected reality that they can understand and interact with.

Title: Modern Computer Vision from CNNs to Foundation Models: A Unified Framework for Representation Learning

Problem Statement
The field of computer vision has evolved rapidly, producing a diverse array of architectures and paradigms including Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), foundation models, and diffusion-based generative systems. While these approaches have achieved state-of-the-art performance in specific domains, the literature remains fragmented. Existing surveys typically focus on isolated model families (e.g., only CNNs or only diffusion models) or specific application domains, failing to provide a unified account of how these paradigms relate to one another. Consequently, the shared mathematical principles underlying modern computer vision—specifically the convergence of discriminative, multimodal, and generative learning—remain obscured by architecture-specific perspectives. The central challenge is to organize this diverse ecosystem into a coherent framework that explains the transition from task-specific architectures to general-purpose visual intelligence.

Methodology
This survey adopts a unified mathematical perspective to analyze the evolution of computer vision. Rather than treating architectures in isolation, the authors establish a common formal framework based on representation learning.

Formal Framework: The paper defines a vision model as a parametric mapping $f_\theta: \mathcal{X} \to \mathcal{Y}$ , where $\mathcal{X}$ is the input image space and $\mathcal{Y}$ is the task-dependent output space. Training is formulated as minimizing empirical risk with a task-dependent loss and regularization. This generic formulation allows for the comparison of diverse models by interpreting them as alternative parameterizations of $f_\theta$ distinguished by their inductive biases and optimization strategies.
Taxonomic Analysis: The survey traces the progression of architectures through this unified lens:
1. CNNs: Analyzed through their layer-wise matrix formulation, highlighting translation equivariance and local connectivity as key inductive biases.
2. Vision Transformers (ViTs): Examined as self-attention-based parameterizations that model global interactions directly, contrasting their lack of strong spatial priors with CNNs.
3. Advanced & Hybrid Architectures: The paper reviews developments that address ViT limitations (e.g., quadratic complexity, data hunger) and the convergence of local and global processing. This includes hierarchical Transformers (e.g., Swin), data-efficient training (e.g., DeiT), and hybrid CNN-Transformer models (e.g., CoAtNet, MaxViT, CvT) that integrate convolutional inductive biases with global attention.
4. Foundation & Multimodal Models: The survey analyzes self-supervised learning (DINO family) and multimodal alignment (CLIP, ImageBind, LLaVA), demonstrating how large-scale pretraining produces transferable representations across tasks and modalities.
5. Generative Frameworks: The paper reviews diffusion models, Diffusion Transformers (DiT), and Rectified Flow, interpreting them as learning probabilistic reverse processes or deterministic transport ODEs within latent spaces.

Key Contributions
The paper makes four primary contributions:

Architectural Evolution: It reviews the progression from CNNs and ViTs to hybrid models, explicitly highlighting the convergence of local (convolutional) and global (attention-based) representation learning mechanisms.
Foundation and Multimodal Models: It analyzes the role of self-supervised (DINO, DINOv2, DINOv3) and multimodal foundation models (CLIP, ImageBind, LLaVA) in the emergence of general-purpose visual intelligence systems.
Generative Frameworks and Future Directions: It reviews diffusion models, DiTs, and Rectified Flow, discussing open challenges regarding efficiency, interpretability, and trustworthy deployment.
Unified Formal Perspective: The core contribution is the synthesis of these diverse paradigms within a single representation-learning framework. The authors demonstrate that CNNs, ViTs, self-supervised models, multimodal systems, and generative architectures all operate on the common principle of constructing, transforming, aligning, or generating latent representations.

Results and Synthesis
The survey does not present new experimental results but synthesizes existing literature to reveal a clear trend: the boundaries between discriminative, multimodal, and generative paradigms are becoming progressively less distinct.

Convergence of Mechanisms: The analysis shows that modern architectures increasingly combine complementary strengths. For instance, hybrid models (CoAtNet, MaxViT) integrate the data efficiency and locality of CNNs with the global context modeling of Transformers.
Unification of Learning Objectives: Self-supervised methods (DINO) and multimodal models (CLIP, ImageBind) are shown to learn representations that generalize across tasks and modalities, moving beyond single-task optimization.
Generative-Discriminative Integration: The paper highlights how generative models (Diffusion, DiT) increasingly utilize Transformer backbones and latent spaces, blurring the line between representation learning and high-fidelity generation.
Mathematical Consistency: By applying the same notation and formal definitions across all sections, the paper demonstrates that these seemingly distinct fields share underlying mathematical principles, such as the manipulation of latent spaces $z = h_\theta(x)$ and the optimization of specific loss functions (contrastive, distillation, denoising).

Significance and Claims
The authors claim that this survey addresses a critical gap in the literature by providing a unified mathematical perspective on modern computer vision.

Reframing the Field: The paper argues that modern computer vision should no longer be viewed as a collection of isolated paradigms but as an evolving ecosystem of closely related approaches. The primary shift is from task-specific architectures toward increasingly general representation-learning systems.
Foundation for Future Research: By identifying the shared principles connecting architectural design, self-supervised learning, multimodal alignment, and generative modeling, the survey provides a conceptual foundation for understanding the emergence of general-purpose visual intelligence.
Future Directions: The paper outlines that the future of the field lies in scalable foundation models that integrate local and global processing, self-supervision, multimodal alignment, and generative capabilities. However, it modestly notes that significant challenges remain, including data efficiency, computational scalability, interpretability, and the need for trustworthy deployment in safety-critical environments.

In summary, the paper posits that the evolution from CNNs to Foundation Models is not a series of disjointed replacements but a continuous convergence toward unified representation learning, where the distinction between "seeing," "reasoning," and "generating" is increasingly integrated within common mathematical frameworks.