🤖 AI

A Scalable Vector Graphics Latent Space

This paper introduces SLS, a Transformer-based autoencoder that establishes a robust, invertible, and continuous latent space for Scalable Vector Graphics by learning compact, fixed-size representations of individual SVG paths that enable high-fidelity reconstruction, efficient similarity search, and significant computational savings compared to token-based approaches.

Original authors: Leonardo Zini, Elia Frigieri, Lorenzo Baraldi

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Leonardo Zini, Elia Frigieri, Lorenzo Baraldi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

For decades, computers have mastered the art of understanding pictures made of pixels, the tiny colored squares that form every photograph on a screen. This success relies on a clever trick: compressing a complex image into a single, dense point of data that captures its essence, allowing machines to find similar pictures, describe them in words, or even create new ones from scratch. However, a different kind of visual language has long remained out of reach for these same tools. This is the world of vector graphics, the mathematical instructions used to draw icons, logos, and user interface elements that stay sharp at any size. Unlike a photograph, which is a fixed grid of dots, a vector image is a set of precise commands telling a computer how to draw lines, curves, and shapes. For years, artificial intelligence struggled to make sense of these instructions, often treating them as clumsy text or blurring them back into pixels, losing the very structure that makes them editable and scalable.

A team of researchers at the University of Modena and Reggio Emilia has now built a bridge to this missing world. They introduced a new system called SLS, which creates a compact, continuous space for these vector instructions, much like the systems that already exist for photographs. Instead of forcing a computer to read thousands of individual drawing commands as a long, confusing sentence, SLS learns to condense each distinct shape into a single, dense point of data. This point holds all the necessary information about the shape's geometry and its style, such as its color and thickness. The system is designed to be reversible; just as a human can look at a drawing and describe it, the computer can take this single data point and perfectly reconstruct the original drawing instructions, down to the last detail. This breakthrough allows machines to search for specific shapes, describe complex icons in natural language, and combine different elements with a speed and efficiency that was previously impossible.

The core of this achievement lies in how the researchers taught the computer to read the instructions. Traditional methods often tried to treat every drawing command as a separate word, leading to sequences that were too long and messy for modern AI to handle effectively. The team instead used a data-driven approach to break the instructions into meaningful chunks, similar to how a human might recognize a phrase rather than spelling out every letter. They trained a neural network to convert these chunks into a unified vocabulary that includes the commands, the numbers defining the curves, and the style settings like opacity and fill color. By processing the data this way, the system learns to map every individual shape to a specific location in a multi-dimensional space. Remarkably, the researchers found that these locations naturally settle into a consistent pattern, forming a sphere where every point is equidistant from the center. This geometric regularity allows the system to perform simple mathematical operations to find similar shapes or mix different concepts together, all without needing to retrain the model for each new task.

The results of this new approach are striking in their efficiency and accuracy. When tested on the task of describing vector images, the system reduced the amount of data the computer needed to process by more than one hundred and fifty times compared to previous methods that treated the instructions as raw text. Despite this massive compression, the system did not lose quality. It successfully generated accurate descriptions of icons and diagrams, outperforming older models that relied on converting the images into pixels first. In tests involving the retrieval of specific shapes from a database of hundreds of thousands of items, the system was able to find the correct matches with a level of precision that surpassed both pixel-based encoders and earlier vector-specific attempts. The researchers also demonstrated that the system could handle images it had never seen before, maintaining its ability to recognize and reconstruct shapes even when the data came from a different source, proving that it had learned the fundamental rules of vector graphics rather than just memorizing specific examples.

Perhaps the most significant aspect of this work is its ability to preserve the original instructions. Many previous attempts to understand vector graphics involved converting them into pixels, which meant the computer could never get the original drawing commands back. If a machine wanted to edit the image, it had to start from scratch. The new system, however, is fully invertible. It can take a complex shape, compress it into a single data point, and then expand that point back into the exact original set of drawing commands. This means the computer can not only understand the image but also manipulate it with the same precision a human designer would expect. The system proved robust against small errors or noise, maintaining the integrity of the shape even when the data was slightly disturbed. By establishing a reliable, compact, and reversible way to represent vector graphics, this work opens the door for a new generation of tools that can generate, search, and edit scalable visual content with the same ease that we currently handle photographs.

Technical Summary: A Scalable Vector Graphics Latent Space (SLS)

Problem Statement
Scalable Vector Graphics (SVG) are the standard for resolution-independent visual content, yet the deep learning community lacks a continuous, dense, and invertible latent space specifically designed for vector representations. Existing approaches suffer from fundamental limitations:

Rasterization: Methods that convert SVGs to pixel-based images (e.g., using CLIP or DINOv2) discard the underlying symbolic geometric structure and style attributes, making them non-invertible to valid SVG code.
Token-Based LLMs: Approaches treating SVG markup as raw text sequences (e.g., IconShop, LLM4SVG) generate or consume long token sequences. This results in sparse, high-dimensional representations that strain context windows, complicate downstream tasks like retrieval, and fail to provide a compact latent space for vector reasoning.
Limited Embedding Quality: Prior vector-specific encoders (e.g., DeepSVG) often rely on rigid, command-based parameterizations that omit stylistic attributes or fail to learn high-fidelity, continuous embeddings.

The core challenge is to construct a continuous embedding space where individual SVG paths map to single dense vectors that preserve both geometric structure and style, are invertible to valid SVG markup, and support efficient vector-space operations.

Methodology: SLS (SVG Latent Space)
The authors propose SLS, a Transformer-based autoencoder designed to learn compact latent representations of individual SVG paths. The architecture and training strategy are defined by the following key components:

Path-Level Granularity: Unlike methods processing entire SVG files, SLS operates on individual paths. Compound SVG objects are decomposed into independent path sequences, reducing average sequence lengths by over two orders of magnitude compared to whole-image approaches.
Data-Driven BPE Tokenization: Instead of using rigid command-based boundaries, SLS employs a Byte Pair Encoding (BPE) tokenizer trained directly on path corpora. This creates a unified vocabulary encompassing drawing commands, numeric coordinates, and style attributes (fill, stroke, opacity). This approach naturally handles the heterogeneous mix of geometric and stylistic information, capturing variations that rigid tokenizers miss.
Encoder-Decoder Architecture:
- Encoder: A Transformer encoder processes the tokenized path sequence. Rather than using a learnable [CLS] token, the model utilizes the final hidden state corresponding to the End-of-Sequence (</s>) token as the global dense embedding ( $\mathbf{z}$ ).
- Decoder: A lightweight autoregressive decoder reconstructs the original token sequence conditioned on the latent vector $\mathbf{z}$ . Due to the deterministic and syntactically strict nature of SVG paths, the decoder employs greedy decoding (selecting the highest probability token at each step) rather than stochastic sampling to ensure valid reconstruction.
Training Objective: The model is trained end-to-end using a token reconstruction loss. To regularize the latent space, Gaussian noise ( $\epsilon \sim \mathcal{N}(0, \sigma^2 \mathbf{I})$ ) is injected into the latent embedding during training. The loss function combines Cross-Entropy ( $\mathcal{L}_{CE}$ ) with a KL-divergence term ( $\mathcal{L}_{KL}$ ) to enforce distributional consistency between target and predicted token distributions.
Hyperspherical Structure: A notable emergent property of the trained encoder is that latent representations exhibit a nearly constant $\ell_2$ -norm. Consequently, embeddings naturally lie close to a unit hypersphere. At inference, embeddings are normalized to unit norm ( $\mathbf{z}_{norm}$ ) to enable efficient cosine-similarity search. For reconstruction, the norm is rescaled by the empirical mean before decoding.

Key Contributions

Invertible Path-Level Latent Space: SLS introduces a scalable latent space where each SVG path is mapped to a single dense vector. This space is fully invertible, allowing the reconstruction of the original SVG path with high geometric and stylistic fidelity—a capability lacking in raster-based encoders.
Unified BPE Tokenization: The paper demonstrates that applying a data-driven BPE tokenizer uniformly to commands, coordinates, and style attributes yields richer representations than rigid, command-based alternatives, effectively capturing stylistic variation.
Hyperspherical Embeddings: The learned space naturally organizes embeddings on a unit hypersphere, enabling efficient downstream operations (retrieval, composition) via simple vector-space operations without complex architectural changes.
Generalization and Efficiency: SLS embeddings generalize across diverse tasks (retrieval, captioning) and significantly reduce computational costs. By compressing thousands of tokens into a single embedding, the approach reduces training FLOPs by over 150× compared to token-based approaches, with specific experiments showing a 167× reduction in training TFLOPs.

Experimental Results
The authors validate SLS on reconstruction, captioning, and retrieval tasks:

Reconstruction: SLS outperforms the state-of-the-art DeepSVG baseline significantly across image-level metrics (MSE, SSIM, LPIPS, DINO similarity) and path-level metrics (BLEU, METEOR, mIoU). Qualitative results confirm superior reconstruction of both geometry and style.
SVG Image Captioning: When used as a visual encoder for Large Language Models (LLMs), SLS embeddings achieve superior performance compared to raw XML token inputs and DeepSVG. Across three LLM backbones (Qwen3, Llama 3.2, Gemma 2), SLS improves CLIP-Score by +2.82 and BLEU5 by +19.19 on average. Crucially, it reduces the average context length from ~2432 tokens to ~15.56 tokens, yielding a 156× speedup in context processing and a 167× reduction in training TFLOPs.
Path Retrieval: In a large-scale retrieval benchmark, SLS outperforms both raster-based encoders (CLIP, DINOv2) and SVG-native baselines (DeepSVG) in visual similarity (MSE-Sim, LAB distance) and syntactic fidelity (BLEU, METEOR), demonstrating its ability to capture structural variations better than image-level models.
Robustness: The latent space remains stable under Gaussian noise (up to $\sigma=0.02$ ) and angular perturbations (up to 30°), maintaining reconstruction quality.
Generalization: SLS shows moderate degradation when applied to out-of-domain datasets (UniSVG), indicating it learns general SVG properties rather than overfitting. Furthermore, aggregating path embeddings allows for effective image-level classification, nearly doubling the performance of DeepSVG.

Significance
The paper posits that SLS provides a foundational building block for vector graphics research, analogous to the role Variational Autoencoders played for raster images. By establishing a continuous, invertible, and semantically structured latent space, SLS bridges the gap between symbolic vector representations and modern foundation models. It enables scalable reasoning over vector content without discarding geometric structure or incurring the computational overhead of processing long token sequences, thereby facilitating future research in vector-native modeling, generation, and editing.

Technical Summary: A Scalable Vector Graphics Latent Space (SLS)

More like this