💻 computer science

MiraGe: Editable 2D Images using Gaussian Splatting

MiraGe is a novel method that enables high-quality, realistic, and physics-based editing of 2D images by representing them as flat-controlled Gaussians in 3D space, overcoming the immutability limitations of previous Gaussian-based image representations.

Original authors: Joanna Waczyńska, Tomasz Szczepanik, Piotr Borycki, Sławomir Tadeja, Thomas Bohné, Przemysław Spurek

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Joanna Waczyńska, Tomasz Szczepanik, Piotr Borycki, Sławomir Tadeja, Thomas Bohné, Przemysław Spurek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a photograph. Usually, we think of a photo as a flat sheet of paper covered in tiny colored dots (pixels). If you want to change the photo—say, make a person's arm wave or a lion's mouth close—you have to edit those dots. It's like painting over a canvas; once you paint, the old picture is gone, and you can't easily "un-paint" it or move the arm in 3D space.

The paper introduces a new tool called MiraGe that changes how we think about these photos. Instead of seeing a photo as a flat sheet of dots, MiraGe treats the image like a flat, invisible sheet of glass floating in a 3D room.

Here is how it works, using some simple analogies:

1. The "Triangle Soup" vs. The "Pixel Grid"

Traditional methods (and even a newer method called GaussianImage) try to rebuild a picture by stacking millions of tiny, flat circles (Gaussians) right on top of each other, like a stack of transparent stickers. It works well for looking at the picture, but if you try to bend or twist that stack, it falls apart or looks weird.

MiraGe does something different. It takes those flat circles and turns them into tiny, flat triangles. Imagine taking a sheet of paper and cutting it into thousands of tiny, flat triangles, then floating them in a 3D room. The authors call this a "Triangle Soup."

The Magic: Because these triangles exist in 3D space, you can grab them, rotate them, or push them around just like you would move real objects in a video game.

2. The "Mirror" Trick

One of the biggest problems with 3D is knowing what's "behind" something. If you have a flat sheet of glass and you look at it from the side, you see nothing.

The Solution: MiraGe uses a clever trick involving mirrors. Imagine you are standing in a room with a photo on the wall. Now, imagine there is a second camera looking at the photo from the other side (like looking through a mirror).
By training the system to see both the front of the photo and its "mirror reflection" at the same time, the computer learns exactly how to arrange those floating triangles so the image looks perfect from any angle. It's like teaching a sculptor to carve a statue by looking at it from two opposite sides simultaneously.

3. Playing with Physics

Because the image is now made of floating triangles in a 3D room, you can connect it to a physics engine (the same kind of software used to make realistic explosions or falling water in movies).

The Result: You can make the photo "react" to the world. You could blow on the image (in the simulation), and the triangles would ripple like a piece of paper in the wind. You could drop a virtual rock on it, and it would crumple realistically.
Why it matters: This isn't just a video effect; the paper claims this allows for "physics-based modification." You aren't just drawing a crumple; you are simulating the actual physics of the image bending.

4. The "3D Illusion" on a 2D Screen

The coolest part is that even though you are doing all this 3D manipulation, the final result is still a standard 2D image.

Analogy: Think of it like a puppet show. The puppets (the triangles) are moving in 3D space behind a screen. When you look at the screen, you see a 2D picture. But because the puppets moved in 3D, the picture on the screen changes in a way that looks incredibly natural and deep, like a 3D effect, even though it's just a flat image.

Summary of What They Claim

Better Quality: They tested MiraGe against other methods (like GaussianImage and neural networks) and found it produces much sharper, clearer images (higher PSNR scores).
Real Editing: Unlike previous methods that just "reconstruct" an image, MiraGe lets you actually edit it. You can move parts of the image, change the perspective, or make it interact with physics, and it looks real.
No Magic Generation: Unlike some AI tools that "guess" what a missing part of a face looks like (generative AI), MiraGe is based on geometry. It moves the actual pieces of the image you already have, preserving fine details (like a dimple on a cheek) without accidentally changing other parts of the face.

In short, MiraGe takes a flat 2D photo, breaks it into a 3D "soup" of triangles, uses mirrors to teach the computer how to arrange them perfectly, and then lets you play with that soup using physics to create realistic, editable images.

Technical Summary: MiraGe – Editable 2D Images using Gaussian Splatting

Problem Statement

Implicit Neural Representations (INRs) and recent Gaussian-based methods like GaussianImage have demonstrated success in encoding 2D images through continuous functions or Gaussian components, achieving high reconstruction quality and compression. However, a significant limitation exists: these methods primarily treat images as static data representations. They lack the capacity for intuitive, user-driven image modification, particularly those mimicking human perception of 2D objects (like photographs) within a 3D spatial context. While 3D Gaussian Splatting (3DGS) allows for scene manipulation, existing 2D Gaussian approaches (e.g., GaussianImage) do not support the transformation of scene objects or the integration of physics-based interactions, limiting their applicability in dynamic editing and animation scenarios.

Methodology

MiraGe addresses these limitations by reframing 2D image encoding through the lens of human visual perception, treating a 2D image as a flat object situated within a 3D space. The core methodology involves the following components:

1. Flat 3D Gaussian Representation

Instead of using standard 3D Gaussians or 2D Gaussians, MiraGe employs flat 3D Gaussians. These are Gaussian components where one scale parameter is forced to zero ( $s_1 = \epsilon$ ), effectively creating a planar distribution.

GaMeS Parametrization: The paper utilizes the GaMeS (Gaussian Mesh Splatting) parametrization to represent these flat Gaussians as triangle-face meshes ("triangle soup"). A Gaussian component $N(m, R, S)$ is mapped to a triangle $V = [v_1, v_2, v_3]$ , where vertices are derived from the mean and scaling/rotation parameters. This allows for direct manipulation of the Gaussian properties (position, scale, rotation) by transforming the underlying triangle mesh.

2. Spatial Configuration and Initialization

The 2D image is initialized on the XZ plane with the center at the origin $(0,0,0)$ . The distance from the camera is a hyperparameter that controls the perceived scale. The initialization range is calculated based on the camera's field of view to ensure the entire image remains visible under perspective projection.

3. Three Control Variants

The paper proposes three distinct approaches for controlling the spatial positioning of these Gaussians:

Amorphous-MiraGe: Gaussians are initialized on the XZ plane but are allowed to move freely (amorphously) in 3D space during training. While this enables 3D manipulation, the authors note it can produce significant artifacts and is insufficient for high-quality representation without further constraints.
2D-MiraGe: To prevent artifacts, Gaussians are strictly anchored to the XZ plane. The method uses a rotation matrix to align Gaussian components with the plane, ensuring orthogonal projection. While effective for 2D, this approach fails when attempting 3D modifications, as the Gaussians appear randomly based on camera position.
Graphite-MiraGe: This variant allows Gaussians to leave the XZ plane (specifically along the Y-axis) by introducing a trainable translation parameter $\gamma$ . This enables 3D modifications while maintaining order relative to camera positions, facilitating complex spatial reasoning and image combination.

4. Mirror Camera Setup

To enhance reconstruction fidelity and control, MiraGe employs a mirror camera setup. Two opposing cameras are positioned along the Y-axis, symmetrically aligned and directed toward each other.

One camera reconstructs the original image $I$ .
The second camera reconstructs the mirror reflection $M(I)$ (horizontally flipped).
The loss function combines the reconstruction costs of both views: $L(I) + L(M(I))$ . This setup ensures Gaussians remain confined within a specific spatial region, improving the accuracy of the representation.

5. Physics Engine Integration

MiraGe integrates with physics engines (Taichi_elements for 2D, Blender for 3D) to enable physics-based modifications. By treating the physics engine as a black box, the model can simulate realistic interactions, deformations, and movements, allowing for dynamic animations of the 2D image content.

Key Contributions

The paper outlines the following primary contributions:

MiraGe Model: Introduction of a novel model that represents 2D images using flat 3D Gaussian components, achieving state-of-the-art reconstruction quality.
3D Spatial Manipulation: The ability to manipulate 2D images within a 3D space, creating the illusion of 3D effects and enabling intuitive perspective adjustments that are difficult in traditional 2D frameworks.
Physics Integration: The successful integration of MiraGe with physics engines, enabling physics-based modifications and interactions for both 2D and 3D environments, enhancing realism in animations.

Results

Quantitative evaluations on the Kodak and DIV2K datasets demonstrate that MiraGe outperforms existing baselines, including classical INRs (SIREN, WIRE, I-NGP, NeuRBF), standard 3DGS, and GaussianImage.

Reconstruction Quality: MiraGe achieves significantly higher PSNR and MS-SSIM scores. For instance, on the DIV2K dataset, MiraGe-100k achieves a PSNR of 54.54 and MS-SSIM of 0.9998, compared to GaussianImage-100k's 41.48 and 0.9981, respectively.
Visual Fidelity: Visual comparisons show that MiraGe produces fewer artifacts and better preserves fine details (e.g., facial dimples, lion's mouth) during local editing compared to generative models like DragGAN or DragDiffusion.
Editing Capabilities: The model supports realistic modifications, such as bending, rotating, and animating 2D images in 3D space, as well as seamless integration of multiple images (collage-like compositions) on distinct planes.

Significance and Claims

The authors claim that MiraGe bridges the gap between 2D image representation and 3D spatial understanding. By simulating human perception—viewing a 2D image as a flat object in 3D space—the method enables:

Intuitive Editing: Users can perform complex transformations (affine, bending) directly in the third dimension.
Dynamic Applications: The integration with physics engines allows for the creation of realistic, physics-factual 2D animations, suitable for virtual reality backdrops and interactive spatial interfaces.
Superior Performance: The approach offers better reconstruction quality and more natural modification capabilities than standard INR or GaussianImage approaches, without relying on generative techniques that may alter semantic content unpredictably.

The paper concludes that while the model is limited by the 2D nature of the training data (e.g., occluded background areas when foreground elements are moved), the combination of flat 3D Gaussians, mirror camera training, and physics integration provides a powerful toolset for expanding the possibilities of image editing and animation.