Stylos: Multi-View 3D Stylization with Single-Forward Gaussian Splatting

Stylos is a single-forward 3D Gaussian framework that achieves geometry-aware, view-consistent 3D style transfer on unposed content by leveraging a Transformer backbone with dual attention pathways and a voxel-based 3D style loss, enabling high-quality zero-shot stylization without per-scene optimization or precomputed poses.

The Gist

Imagine you have a collection of photos of a park taken from different angles. You want to turn this real-life park into a 3D world that looks like a Van Gogh painting, but with a catch: you want to do it instantly, without spending hours teaching the computer how to paint, and you want the result to look consistent no matter which angle you look at.

That is exactly what the paper Stylos solves.

Here is the breakdown of how it works, using simple analogies:

1. The Problem: The "Slow Artist" vs. The "Instant Artist"

Previously, if you wanted to turn a 3D scene into a painting, you had to use methods that were like hiring a slow, meticulous artist.

• The Old Way: You'd feed the computer the photos, and it would spend hours (or days) "optimizing" or tweaking the 3D model for that specific scene to make it look like the painting. If you wanted to paint a different scene, you'd have to start the whole long process over again. It was like hiring a painter to repaint a house every time you moved to a new one.
• The Stylos Way: Stylos is like a super-fast, instant translator. You give it a photo of a scene and a photo of a painting style, and poof—it instantly generates a 3D world that looks like that painting. It doesn't need to "learn" the new scene; it just applies the style immediately.

2. The Secret Sauce: The "Two-Lane Highway"

The core of Stylos is a neural network (a type of AI brain) that acts like a two-lane highway for information.

• Lane 1: The Architect (Geometry)
  This lane is dedicated to understanding structure. It looks at your photos and figures out where the walls, trees, and cars are. It uses a "self-attention" mechanism, which is like the architect looking at the whole blueprint to make sure the building stands up straight. It ignores the painting style here; it only cares about the shape and depth.
• Lane 2: The Painter (Style)
  This lane is dedicated to the look. It takes the "style reference" (the Van Gogh painting) and injects those colors and brushstrokes into the scene. It uses a "cross-attention" mechanism, which is like the painter looking at the Architect's blueprint and saying, "Okay, I'll paint the walls yellow and the sky blue, but I'll keep the walls in the exact same shape."

Why this matters: By keeping the "shape" and "color" on separate lanes that talk to each other but don't mix up their jobs, Stylos ensures the 3D object doesn't get distorted just because the colors changed.

3. The Magic Trick: The "Voxel Loss" (The 3D Checksum)

One of the hardest things in 3D art is making sure the painting looks the same from every angle. If you walk around a 3D statue, the paint shouldn't look like it's sliding off or changing patterns randomly.

• The Old Problem: Old methods checked the style like a 2D photo. They looked at one picture at a time. This meant the computer might paint the left side of a tree yellow and the right side blue, not realizing they are part of the same 3D object.
• The Stylos Solution: They invented a "Voxel-Level Style Loss."
  • Imagine taking your 3D scene and slicing it into millions of tiny, invisible 3D cubes (like a giant 3D pixel grid called voxels).
  • Stylos looks at all the different camera angles and "fuses" them into these 3D cubes.
  • It then checks: "Does the paint inside this 3D cube look like the Van Gogh style from every angle?"
  • If the paint looks weird or inconsistent in 3D space, the AI corrects it. This ensures the style is "glued" to the 3D object, not just painted on the surface of a flat image.

4. Why It's a Big Deal

• Zero-Shot Learning: You can show Stylos a style it has never seen before (like a specific abstract art style), and it will apply it perfectly to a new scene without needing to be retrained. It's like a chef who can taste a new spice and immediately know how to cook a whole new dish with it, without a recipe.
• Speed: It works in a "single forward pass." This means it doesn't need to loop through the data to fix mistakes. It sees the input and gives the output instantly.
• Versatility: It works on everything from a single photo of a pizza to hundreds of photos of a city street.

Summary Analogy

Imagine you have a Lego castle (the 3D scene).

• Old methods would take the castle apart, paint every single brick by hand to match a picture, and then try to put it back together. It took forever, and sometimes the bricks didn't fit right.
• Stylos is like a magic spray paint gun. You point it at the Lego castle, show it a picture of a Van Gogh painting, and it instantly sprays the whole castle with the right colors and textures. The bricks stay in their exact 3D positions (the geometry), but the whole thing instantly looks like a masterpiece, no matter which side you look at.

In short: Stylos is the first system that can instantly turn any 3D scene into a consistent, high-quality painting without needing to "study" the scene first.

Technical Summary

1. Problem Definition

The paper addresses the challenge of 3D style transfer (transferring an artistic style from a reference image to a 3D scene) under the constraints of unposed inputs (images without known camera parameters) and zero-shot generalization.

• Current Limitations: Existing methods typically rely on:
  • Per-scene optimization: Requiring costly, time-consuming optimization for every new scene (e.g., NeRF-based or 3DGS-based methods like StyleGaussian).
  • Precomputed poses: Requiring known camera intrinsics/extrinsics.
  • Limited Generalization: Struggling to generalize to unseen categories, scenes, or styles without retraining.
  • Inconsistency: Failing to maintain geometric fidelity and view consistency when applying styles across multiple viewpoints.
• Goal: Develop a single-forward-pass framework that takes unposed multi-view content images and a single style reference image to instantly generate a stylized 3D Gaussian Splatting (3DGS) scene, achieving high visual fidelity, geometric coherence, and cross-view consistency without scene-specific training.

2. Methodology: The Stylos Framework

Stylos is a Transformer-based feed-forward architecture built upon the VGGT (Visual Geometry Grounded Transformer) backbone. It disentangles geometry and style processing into two pathways.

A. Network Architecture

1. Geometric Backbone (VGGT):

   • Uses a pre-trained VGGT backbone to process multi-view content images.
   • Employs alternating attention (frame-wise and global self-attention) to infer geometry-related parameters: 3D positions ($\mu$), scales ($s$), rotations ($r$), opacities ($\alpha$), and camera parameters (poses/intrinsics).
   • This pathway remains self-attention only, ensuring that geometric reasoning is derived solely from the content structure, preserving geometric fidelity.

2. Style Aggregator (Cross-Block Modules):

   • Introduces a dedicated branch to inject style information.
   • Replaces standard Transformer blocks with CrossBlocks, which insert a cross-attention layer between the self-attention and MLP layers.
   • Mechanism: Content tokens act as Queries ($Q$), while style tokens (from the reference image) act as Keys and Values ($KV$).
   • Topological Strategies: The paper evaluates three strategies:
     • Frame CrossBlock: Independent style injection per view (preserves view-specific structure but lacks inter-view consistency).
     • Global CrossBlock: Concatenates all views for global reasoning (best for consistency).
     • Hybrid: A combination of both.
   • The output of this branch predicts spherical harmonic coefficients ($c$) for the Gaussian colors, conditioned on the style.

3. Prediction Heads & Voxelization:

   • Geometry Head: Outputs Gaussian primitives ($\mu, s, r, \alpha$).
   • Style Head: Outputs color coefficients ($c$).
   • Gaussian Adapter: Merges geometry and style outputs into a unified set of 3D Gaussians.
   • Voxelization: Uses a differentiable voxelization step (inspired by AnySplat) to cluster nearby points and fuse features, reducing redundancy and balancing density.

B. Training Strategy

The model is trained in two stages:

1. Stage 1: Geometry Pretraining:

   • Initializes with VGGT weights.
   • Trains end-to-end using a frozen VGGT teacher for pose/depth supervision.
   • Uses a random content view as a pseudo-style reference (with color jitter) to prevent trivial identity mapping and improve robustness.
   • Loss: Reconstruction + Distillation.

2. Stage 2: Stylization Fine-tuning:

   • Freezes all geometry-related modules.
   • Updates only the Style Aggregator and Color Head.
   • Uses real artistic style images for conditioning.
   • Losses: Reconstruction + Content Loss + CLIP Loss + TV Regularizer + 3D Voxel Style Loss.

C. Key Innovation: 3D Voxel-Based Style Loss

Standard 2D style losses (matching Gram matrices or BN statistics per image) fail to enforce 3D consistency. Stylos proposes a Voxel-level 3D Style Loss:

• Process: Multi-view rendered features are unprojected and fused into a discrete 3D voxel grid.
• Mechanism: Computes Batch Normalization statistics (mean and variance) directly on the aggregated 3D voxel features and aligns them with the statistics of the style reference image.
• Benefit: Explicitly enforces style consistency across views and the underlying 3D structure, preventing "flickering" or inconsistent textures between viewpoints.

3. Key Contributions

1. Single-Forward 3D Stylization: A feed-forward framework that generates stylized 3D Gaussians from unposed inputs in a single pass, eliminating per-scene optimization.
2. Dual-Pathway Architecture: A design where geometry relies on self-attention (for structural integrity) and style is injected via cross-attention (for visual consistency), effectively decoupling shape and appearance.
3. Voxel-Level 3D Style Loss: A novel loss function that operates in 3D voxel space to enforce global style consistency and geometric coherence, outperforming traditional 2D image-level losses.
4. Zero-Shot Generalization: Demonstrates robust performance across unseen categories, scenes, and styles, scaling from single views to hundreds of views.

4. Experimental Results

The authors evaluated Stylos on CO3D (cross-category), DL3DV-10K, and Tanks & Temples (cross-scene) datasets, using WikiArt and DELAUNAY for style references.

• Quantitative Performance:
  • Consistency: Stylos achieved the best scores in short-range and long-range LPIPS and RMSE across all tested scenes, significantly outperforming baselines like StyleGaussian, G-Style, SGSST, and Styl3R.
  • Artistic Quality: Achieved top-tier ArtScore and ArtFID scores, indicating high-quality style transfer.
  • Efficiency: Inference time is 0.05 seconds per scene (single-forward), compared to minutes or hours for per-scene optimization methods.
• Qualitative Performance:
  • Geometry Preservation: Successfully maintained fine structural details (e.g., pizza crust, skateboard edges) while applying complex styles.
  • View Consistency: Eliminated artifacts and color inconsistencies common in other methods (e.g., the "blue truck door" issue in Styl3R).
  • Ablation Studies: Confirmed that Global CrossBlock designs and 3D Voxel Loss are critical for high-quality results. Frame-only CrossBlocks led to blurred edges, and Image-level losses failed to transfer style to the whole scene.

5. Significance and Impact

• Real-Time 3D Content Creation: By removing the need for per-scene optimization, Stylos makes high-fidelity 3D stylization feasible for real-time applications in VR/AR, gaming, and digital content creation.
• Robustness to Unposed Data: The ability to work with unposed, uncalibrated multi-view images makes the method highly practical for real-world scenarios where camera parameters are unknown.
• Bridging 2D and 3D Style Transfer: The introduction of the voxel-based loss provides a new paradigm for enforcing 3D consistency in generative tasks, moving beyond 2D image statistics to true volumetric alignment.
• Scalability: The framework scales effectively from single views to large-scale multi-view collections, demonstrating the potential for processing massive datasets in a single forward pass.

In conclusion, Stylos represents a significant leap forward in 3D stylization, offering a fast, generalizable, and geometrically consistent solution that overcomes the primary bottlenecks of previous optimization-based approaches.