💻 computer science

Soft Anisotropic Diagrams for Differentiable Image Representation

Soft Anisotropic Diagrams (SAD) is a new differentiable image representation that uses adaptive, anisotropic, and additively weighted Voronoi partitions to achieve superior compression, faster training speeds, and efficient rendering compared to existing state-of-the-art methods.

Original authors: Laki Iinbor, Zhiyang Dou, Wojciech Matusik

Published 2026-04-27

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Laki Iinbor, Zhiyang Dou, Wojciech Matusik

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to recreate a beautiful, complex painting using only a limited number of colored stickers.

If you use big, round, blurry stickers (like a "Gaussian" approach), the painting looks smooth, but the edges of objects look fuzzy and out of focus. If you try to use tiny, sharp squares (like a "pixel" approach), you need millions of them, which takes up too much storage and is slow to process.

The paper "Soft Anisotropic Diagrams (SAD)" introduces a clever new way to "sticker" an image that is both incredibly fast and incredibly sharp.

The Core Idea: The "Smart Sticker"

Instead of using standard round stickers, SAD uses "Smart Stickers" (called sites). These stickers have three special powers:

The Shape-Shifter (Anisotropy): Unlike a round sticker, a SAD sticker can stretch. If it’s placed on a long, straight edge (like the side of a building), it stretches into a long, thin needle shape to perfectly cover that edge. If it’s in a wide-open sky, it stays more compact.
The Volume Knob (Temperature): Each sticker has a "temperature" setting. A "hot" sticker has very hard, sharp edges—it says, "This pixel is mine, and no one else's!" A "cool" sticker is soft and blurry, perfect for blending colors in a sunset.
The Neighborhood Watch (Top-K Propagation): To make sure the computer doesn't get overwhelmed trying to check every single sticker for every single pixel, the system uses a "neighborhood watch" rule. Each pixel only looks at its 8 closest "competitive" stickers. This makes the math incredibly fast, like only checking your immediate neighbors instead of the whole city.

A Creative Analogy: The "Stained Glass Window"

Think of a traditional image like a photograph. Now, think of SAD like a Stained Glass Window being built by a master craftsman:

The Glass Pieces (The Sites): Instead of just blobs of color, the craftsman cuts pieces of glass into custom shapes. If there is a long diagonal line in the design, he cuts a long, thin shard of glass to fit it perfectly.
The Lead Strips (The Boundaries): In a real stained glass window, the lead lines create sharp, clear boundaries. SAD does this mathematically. It uses "temperature" to decide where the lines should be razor-sharp (to define an object) and where they should be soft and blended (to show a shadow).
The Fast Assembly (The GPU Optimization): Instead of the craftsman wandering aimlessly around the workshop, he has a system where each section of the window is assembled by a small team working only on their local area. This allows the window to be built at lightning speed.

Why does this matter?

In the real world, we are constantly trying to shrink the size of images and videos (compression) so they load faster on our phones, while keeping them looking crystal clear.

The "SAD" method wins because it hits the "Goldilocks Zone":

It’s much faster: It can "learn" what an image looks like much quicker than previous methods (up to 19 times faster!).
It’s much sharper: Because the stickers can stretch and sharpen their edges, it captures fine details (like hair or text) that other methods blur away.
It’s efficient: It uses very little "memory" (bits) to describe the image, making it a potential game-changer for how we store and stream high-quality visuals.

In short: SAD is like giving a computer a box of magical, shape-shifting, temperature-controlled stickers that allow it to rebuild any picture perfectly, using the fewest pieces possible, in a fraction of the time.

Technical Summary: Soft Anisotropic Diagrams (SAD) for Differentiable Image Representation

Soft Anisotropic Diagrams (SAD) is a novel, explicit, and differentiable image representation method designed to bridge the gap between implicit neural representations (which lack structural explicitness) and splat-based representations (which suffer from kernel overlap and high encoding costs).

1. The Problem: The Efficiency-Structure Trade-off

Current differentiable image representations face a fundamental tension:

Implicit Neural Representations (e.g., Instant-NGP): Highly expressive and continuous but lack explicit spatial ownership. This makes localized editing, pruning, and budget control (compression) difficult and indirect.
Splat-based/Gaussian Representations (e.g., Image-GS): Offer explicit primitives but rely on kernel overlap to represent signals. This overlap blurs per-pixel responsibility, complicates budget management, and leads to high computational costs due to expensive tile-binning and atomic-heavy gradient accumulation.
The Encoding Gap: There is a persistent "encoding cost" problem where fitting a compact, high-quality representation is often orders of magnitude slower than evaluating it.

2. Methodology: Soft Anisotropic Voronoi Partitioning

SAD replaces overlapping kernels with a learnable soft partition of unity based on a generalized Voronoi construction (specifically, an Apollonius diagram).

A. Site Parameterization:
The image is represented by $N$ adaptive sites. Each site $i$ is defined by:

Position ( $p_i$ ): 2D coordinates.
Color ( $c_i$ ): The RGB value.
Anisotropy ( $u_i, a_i$ ): A direction and a log-anisotropy scalar that allows the site's "influence" to stretch into elongated shapes (ideal for edges).
Radius ( $r_i$ ): An additive weight that controls the site's reach.
Temperature ( $\tau_i$ ): A per-site parameter that controls the "hardness" of the boundary. High $\tau$ creates sharp, crisp edges; low $\tau$ creates smooth, soft gradients.

B. Rendering via Softmax Blend:
Instead of summing Gaussians, SAD computes a distance score $d_{mix}$ for each pixel relative to its nearest sites. A pixel's color is a softmax blend over its top- $K$ nearest sites. This ensures that every pixel has explicit "ownership" by a small, fixed number of primitives.

C. Top- $K$ Propagation (The GPU-First Engine):
To avoid the $O(N)$ cost of checking every site for every pixel, SAD maintains a per-pixel list of the $K$ most competitive sites. It uses a Jump Flooding Algorithm (JFA) inspired scheme to propagate site IDs across the image. This allows for:

Constant Query Cost: Rendering and gradient computation scale with $O(P \cdot K)$ (pixels $\times$ $K$ ), making it highly regular and bandwidth-efficient for GPUs.
Temporal Reuse: The list is updated incrementally during training, significantly speeding up convergence.

D. Adaptive Budget Control:
SAD manages its complexity through an error-density heuristic. It identifies sites that explain little "mass" but contribute high residual error, triggering densification (splitting sites). Conversely, it uses a removal-delta score to prune sites that, if deleted, would minimally impact reconstruction quality.

3. Key Contributions

New Geometric Framework: Introduces a differentiable, anisotropic, additively weighted Voronoi-style partition for images.
Efficient Algorithm: Develops a GPU-optimized Top- $K$ propagation and a tiled threadgroup hash reduction for gradient accumulation, minimizing atomic contention.
Adaptive Pipeline: A complete optimization loop featuring content-adaptive initialization, densification, and pruning.

4. Results and Performance

SAD demonstrates superior performance across multiple benchmarks:

Reconstruction Quality: Consistently outperforms Image-GS and Instant-NGP at matched bitrates. On the Kodak dataset, it achieves 46.0 dB PSNR, significantly higher than baselines.
Training Speed: Achieves 4–19× end-to-end speedups over state-of-the-art methods. For example, on Kodak, it encodes in 2.2s compared to 28s for Image-GS.
Structural Adaptivity: Ablation studies prove that learnable anisotropy and temperature are critical; anisotropy allows cells to align with edges, while temperature allows the model to switch between smooth textures and sharp boundaries.
Versatility: Demonstrated effectiveness in solving differentiable PDEs (where explicit site boundaries allow for easy enforcement of hard Dirichlet constraints) and 1D signal fitting (where it handles discontinuities better than SIREN or Gaussians).

5. Significance

SAD represents a significant step toward high-throughput, high-quality differentiable image compression. By moving from "overlapping blobs" to "structured partitions," it provides a mathematically grounded way to represent images that is both computationally efficient for modern GPUs and structurally intuitive for downstream tasks like editing, physics simulation, and adaptive-resolution rendering.