💻 computer science

Less Gaussians, Texture More: 4K Feed-Forward Textured Splatting

The paper introduces LGTM, a feed-forward framework that achieves scalable 4K novel view synthesis by decoupling geometric complexity from rendering resolution through compact Gaussian primitives coupled with per-primitive textures, thereby overcoming the quadratic primitive growth limitations of existing methods.

Original authors: Yixing Lao, Xuyang Bai, Xiaoyang Wu, Nuoyuan Yan, Zixin Luo, Tian Fang, Jean-Daniel Nahmias, Yanghai Tsin, Shiwei Li, Hengshuang Zhao

Published 2026-03-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yixing Lao, Xuyang Bai, Xiaoyang Wu, Nuoyuan Yan, Zixin Luo, Tian Fang, Jean-Daniel Nahmias, Yanghai Tsin, Shiwei Li, Hengshuang Zhao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a massive, hyper-realistic 3D city out of tiny, glowing marbles. This is essentially what 3D Gaussian Splatting does to create virtual worlds for video games, VR, and movies. Each "marble" (or Gaussian) holds a piece of the scene's geometry and color.

The problem with current methods is that they are inefficient. To make a city look sharp and detailed (like 4K resolution), you need to throw millions of these marbles at the wall. It's like trying to paint a masterpiece by gluing individual grains of sand to a canvas. As you try to make the image bigger (higher resolution), the number of marbles you need explodes, crashing your computer's memory.

Enter LGTM (Less Gaussians, Texture More). Think of LGTM as a smart architect who realizes: "Why glue millions of grains of sand when we can use fewer, bigger tiles and paint the details on them?"

Here is how LGTM works, broken down into simple concepts:

1. The Old Way: "More Marbles, More Problems"

Imagine you are trying to recreate a photo of a brick wall.

The Old Method: You use a million tiny marbles. Each marble is a different shade of red or brown. To get a sharp 4K image, you need so many marbles that your computer runs out of space and crashes. It's like trying to fill a swimming pool with thimbles.
The Limitation: If you want to double the image size, you need 4 times as many marbles. If you want 4K, you need 64 times as many. It becomes impossible.

2. The LGTM Solution: "Fewer Marbles, Better Paint"

LGTM changes the rules. Instead of making every marble a different color, it uses two separate teams to build the scene:

Team A (The Geometers): They build the shape of the scene using a small, compact set of marbles. They figure out where the walls, floors, and objects are. They don't worry about the fine details yet; they just get the structure right.
Team B (The Artists): They take those few marbles and wrap them in high-definition "wrapping paper" (textures). Instead of the marble itself being red, the paper wrapped around it has a detailed brick pattern painted on it.

The Magic Trick:
Because the "shape" (geometry) and the "look" (texture) are separated, LGTM can use fewer marbles to define the shape, but huge, detailed textures to define the look.

Analogy: Imagine a low-resolution 3D model of a car.
- Old Way: You need millions of pixels to make the car look shiny and detailed.
- LGTM Way: You build a simple, smooth clay model (few marbles) and then project a high-resolution photo of a real car onto it. The result looks like a real car, but you used way less "clay."

3. Why This is a Big Deal (The "Feed-Forward" Part)

Usually, to get a perfect 4K image, computers have to spend hours "optimizing" the scene for every single new picture. It's like a chef tasting and adjusting a soup for every single guest.

LGTM is Feed-Forward. This means it's like a fast-food kitchen.

You hand it a photo (or two).
It instantly spits out a 4K 3D scene in a split second.
No waiting, no re-tuning, no "per-scene optimization." It just works immediately.

4. The Results: From "Blurry" to "Cinema Quality"

The paper shows that LGTM can create 4K resolution images (which are incredibly sharp) using the same amount of computer memory that other methods need for just a blurry, low-resolution image.

Before: To get 4K, you needed a supercomputer, and it would take forever.
Now: LGTM can do it on a standard high-end gaming GPU in a blink of an eye, using 64 times fewer geometric primitives (marbles) than before.

Summary Analogy

Think of building a house:

Old Methods: You try to build the house by stacking millions of individual Lego bricks. To make the house bigger, you need millions more bricks. It takes forever and runs out of space.
LGTM: You build the house's frame with a few large wooden beams (the Gaussians). Then, you wrap the whole thing in a high-resolution wallpaper that looks exactly like wood grain, brick, or paint (the Textures). You get a beautiful, detailed house instantly, using very few structural pieces.

In short: LGTM is the first tool that lets us instantly generate high-definition, 4K 3D worlds without needing a supercomputer or hours of waiting time, simply by being smarter about how we separate "shape" from "skin."

1. Problem Statement

Existing feed-forward 3D Gaussian Splatting (3DGS) methods face a fundamental scalability bottleneck regarding resolution.

Quadratic Growth: Current methods predict pixel-aligned primitives. As image resolution increases (e.g., from 512px to 4K), the required number of Gaussian primitives grows quadratically (64× for 4K). This makes high-resolution synthesis computationally prohibitive and memory-intensive.
Coupling of Geometry and Appearance: Standard 3DGS encodes appearance (color) and geometry within a single primitive. To represent high-frequency textures on simple geometric surfaces, an excessive number of Gaussians is required.
Limitations of Prior Art: While "textured Gaussian" methods exist to decouple appearance from geometry, they typically require per-scene optimization, preventing them from being true feed-forward (instant) reconstruction systems.

2. Methodology: LGTM Framework

The authors propose LGTM (Less Gaussians, Texture More), a feed-forward framework that decouples geometric complexity from rendering resolution.

Core Architecture: Dual-Network Design

LGTM utilizes two distinct networks to separate geometry prediction from texture prediction:

Primitive Network ( $f_{prim}$ ):
- Input: Low-resolution images ( $I_{low}$ ).
- Function: Predicts a compact set of geometric primitives (2D Gaussian parameters: position $\mu$ , scale $s$ , rotation $r$ , opacity $o$ , and base color $c$ ).
- Strategy: It operates on low-resolution inputs to keep the primitive count low, regardless of the target output resolution.
Texture Network ( $f_{texture}$ ):
- Input: High-resolution images ( $I$ ) and features from the primitive network.
- Function: Predicts per-primitive texture maps (a color texture $T^c$ and an alpha texture $T^\alpha$ ) for each Gaussian.
- Mechanism:
  - Image Patchification: Extracts local high-frequency features from high-res patches.
  - Projective Mapping: Uses an inverse transformation to "render" the source image back onto the Gaussian primitive's local coordinates, creating a projective prior texture.
  - Feature Aggregation: Fuses backbone features, patch features, and projective features to generate the final texture maps.

Rendering Mechanism

Instead of using a single color value per Gaussian, LGTM renders using bilinear sampling from the predicted texture maps at the ray-splat intersection point:

Alpha: $a_i(u) = o_i \cdot T^\alpha_i[u]$ (replaces standard Gaussian falloff).
Color: $\hat{c}_i(u, d_i) = T^c_i[u] + SH(c_i, d_i)$ (adds high-frequency detail to the Spherical Harmonics base color).

Training Strategy

LGTM employs a staged training recipe to ensure stability:

Stage 1 (Geometry Foundation): Pre-train the primitive network using low-resolution inputs but high-resolution supervision. The network renders low-res primitives at full resolution (e.g., 4K) and computes loss against the ground truth. This forces the network to learn appropriate scales and geometric priors for high-res rendering.
Stage 2 (Texture Integration): Jointly train the primitive and texture networks. The primitive network parameters are frozen or trained with a reduced learning rate to maintain geometric stability, while the texture network learns to add high-frequency details.

3. Key Contributions

First Feed-Forward Textured Gaussians: LGTM is the first method to predict textured Gaussians in a purely feed-forward manner, eliminating the need for per-scene optimization.
Resolution Decoupling: By separating geometry (low-res primitives) from appearance (high-res textures), LGTM achieves 4K novel view synthesis with a fixed, small number of primitives (e.g., $512 \times 288$ ), overcoming the quadratic scaling barrier.
General Applicability: The framework is modular and can be applied to various baseline methods (monocular, two-view, multi-view) and input settings (with or without camera poses).
Efficiency: It significantly reduces memory and compute costs compared to naive high-resolution scaling.

4. Experimental Results

The authors evaluated LGTM on RealEstate10K (RE10K) and DL3DV-10K datasets across single-view, two-view, and multi-view settings.

Performance Metrics: LGTM consistently outperforms baselines (NoPoSplat, DepthSplat, Flash3D, VGGT) in PSNR, SSIM, and LPIPS across all resolutions (1K, 2K, 4K).
- Example (4K, Two-View): LGTM achieved a PSNR of 24.489 vs. 23.022 for the baseline, with a significant LPIPS improvement (0.200 vs. 0.351).
Scalability & Efficiency (Table 4):
- Scaling from 512px to 4K (a 64× increase in pixel count) using LGTM requires only 1.80× the peak GPU memory and 1.47× the total inference time compared to the low-resolution baseline.
- In contrast, traditional methods fail to train or run at 2K/4K due to Out-Of-Memory (OOM) errors.
Ablation Study: Removing components (e.g., projective mapping or texture alpha) leads to significant performance drops, confirming that both the high-res feature extraction and the per-primitive alpha/color textures are essential.

5. Significance and Impact

Enabling 4K Real-Time Rendering: LGTM makes high-fidelity, 4K novel view synthesis feasible in a feed-forward setting, a capability previously restricted to slow, per-scene optimization methods.
Resource Efficiency: It drastically lowers the barrier for high-resolution 3D reconstruction, making it viable for resource-constrained applications like Augmented Reality (AR) and Virtual Reality (VR).
Paradigm Shift: The paper establishes a new paradigm for 3DGS where geometric complexity is kept low, and visual fidelity is achieved through learned texture maps, effectively solving the "resolution vs. primitive count" trade-off.

Limitations: The authors note that reconstruction quality still heavily relies on the accuracy of the underlying geometry (performing best in posed scenarios). Additionally, the texture resolution is currently pre-defined, requiring manual tuning to balance quality and cost.