WildSVG: Towards Reliable SVG Generation Under Real-Word Conditions

This paper introduces the WildSVG Benchmark, comprising real-world and synthetic datasets, to address the lack of evaluation resources for SVG extraction and reveals that while current multimodal models struggle with noisy natural images, iterative refinement methods offer a promising path toward reliable performance.

The Gist

Imagine you have a beautiful, hand-drawn logo on a piece of paper. Now, imagine that paper is crumpled, stained with coffee, sitting on a busy street, and partially covered by a pigeon's shadow.

The Challenge:
Your goal is to take a photo of this messy, real-world scene and magically turn it back into a perfect, clean, digital drawing file (called SVG) that a graphic designer can edit, resize, and use anywhere.

This is the problem the paper "WildSVG" tackles.

The Problem: The "Clean Room" vs. The "Wild"

For a long time, AI models have been really good at drawing vector graphics, but only if you give them a clean, perfect image (like a photo of a logo on a white background) or a text description like "draw a red apple."

But the real world isn't a clean room. It's "wild."

• The Wild: Logos are on billboards in the rain, on t-shirts with wrinkles, or on cars moving fast. They are blurry, distorted, and surrounded by clutter.
• The Failure: When you ask current AI models to look at a messy photo and recreate the logo, they get confused. They might draw the whole background, miss the logo entirely, or create a messy digital sketch that looks nothing like the original.

The Solution: Introducing "WildSVG"

The researchers realized there was no "test" for this specific skill. It's like trying to teach a driver to race in a blizzard, but you've only ever let them practice on a sunny, empty track.

To fix this, they built WildSVG, a new training ground with two types of "driving courses":

1. The "Natural" Course (Real Life): They took thousands of real photos of company logos from the internet (like a Starbucks cup on a rainy sidewalk) and paired them with the perfect digital version of that logo. This is the "messy" test.
2. The "Synthetic" Course (Fake but Hard): They took perfect digital logos and used AI to paste them into realistic, messy scenes (like a logo on a crumpled soda can). This lets them test the AI with specific, known difficulties.

The Experiment: Who is the Best Artist?

The researchers put the world's smartest AI models (like GPT-5, Claude, and Gemini) through this test. They asked the AIs: "Look at this messy photo. Ignore the background. Give me the clean digital code for just the logo."

Here's what they found:

• The "Semantic" Trap: Most AIs are like students who memorized the idea of a logo but can't draw it perfectly. If asked to draw the "Heineken" logo, the AI might write the word "Heineken" in a font that looks kind of like the real one. It gets the "meaning" right (the semantic similarity is high), but the actual drawing is wrong (the pixel accuracy is low).
• The "StarVector" Glitch: One specific model, trained to be a vector expert, got so confused by the messy background that it tried to draw the entire photo instead of just the logo. It forgot to listen to the instructions!
• The "Iterative" Hope: The best results came from models that didn't just guess once. They used a "try, check, and fix" method. The AI draws a draft, looks at it, says "Oops, that curve is wrong," and fixes it. This "iterative refinement" is the most promising path forward.

The Verdict

Currently, even the smartest AIs are like novice artists trying to copy a masterpiece while wearing foggy glasses. They can get the general shape and color, but they struggle with the fine details and the messy real-world context.

Why does this matter?
If we can solve this, we could:

• Turn a photo of a hand-drawn sketch into a professional vector file instantly.
• Help designers find and edit logos from old, messy archives.
• Automate the creation of digital assets from the real world.

In short: The paper says, "We built the first real-world test for AI vector drawing, and while the AI is getting smarter, it still has a long way to go before it can reliably turn our messy world into clean, editable digital art."

Technical Summary

1. Problem Definition

The paper addresses the gap between existing SVG generation capabilities and real-world application requirements.

• Current Limitations: State-of-the-art Multimodal Large Language Models (MLLMs) and vectorization tools (e.g., StarVector) perform well on clean, synthetic renderings or text prompts. However, they fail in real-world scenarios where target graphics (logos, icons) are embedded in noisy, cluttered, and context-rich natural images.
• The Challenge: The task of SVG Extraction requires not just generating vector code, but performing selective abstraction. The model must:
  1. Identify and localize specific graphical elements within a complex scene.
  2. Ignore irrelevant visual content (background, shadows, textures).
  3. Handle domain shifts like perspective distortion, occlusion, and lighting variations.
  4. Generate structured, editable SVG code that faithfully represents the isolated element.
• The Bottleneck: There is a lack of suitable benchmarks and datasets that pair real-world images with ground-truth SVG annotations, preventing systematic evaluation and progress.

2. Methodology: The WildSVG Benchmark

To address the data scarcity, the authors introduced WildSVG, the first benchmark specifically designed for SVG extraction. It consists of two complementary datasets:

A. Natural WildSVG

• Source: Built upon the Logos-in-the-Wild dataset.
• Construction: Real-world images containing company logos are paired with their corresponding vector annotations.
• Process:
  1. Logo detections (bounding boxes) are extracted.
  2. Candidate SVGs are retrieved from public databases (e.g., worldvectorlogo.com).
  3. Validation: A VLM-based judging model and DINOv2 feature similarity scores are used to rank and validate the match between the rasterized SVG and the cropped image, ensuring high-quality ground truth.
• Content: Contains real logos with natural noise, occlusions, and perspective distortions.

B. Synthetic WildSVG

• Source: Built upon the SVG-Stack dataset.
• Construction: Complex SVGs are integrated into realistic scenes using an image generation model (Gemini-2.0-flash-preview-image-generation, now Nano Banana).
• Process:
  1. SVGs and their descriptions are used as prompts to generate synthetic compositions.
  2. Prompts are manually optimized to ensure the SVG is embedded naturally while preserving fidelity.
  3. Bounding boxes are manually annotated for the test split.
• Content: Provides diverse, complex SVG structures under controlled but visually challenging conditions (e.g., varying lighting, backgrounds).

Evaluation Protocol

The benchmark evaluates models using two distinct settings:

1. One-Step Extraction: The model receives the full image and a "focus prompt" specifying the target element.
2. Two-Step Extraction: The image is first cropped using perfect bounding boxes (simulating an ideal detector), and the model generates the SVG from the crop. This serves as an upper bound for pipeline performance.

Metrics:

• Pixel Fidelity: L2 distance, SSIM.
• Perceptual/Semantic Similarity: LPIPS, DINO score.
• Code Quality: Token Length Difference (compactness) and Primitive Diversity (editability proxy).

3. Key Contributions

1. Task Definition: Formalized SVG Extraction as a distinct research problem, differentiating it from full-image vectorization and logo detection.
2. WildSVG Dataset: Created the first dataset pairing natural images with ground-truth SVG annotations, comprising both Natural (real photos) and Synthetic (generated scenes) splits.
3. Comprehensive Benchmarking: Evaluated a wide range of state-of-the-art models, including:
   • General VLMs: GPT-5, Claude Opus 4.1, Gemini 2.5, GLM 4.5V, Qwen2.5VL.
   • Specialized Models: Qwen2.5VL-8B SVG RL (trained with StarVector pipeline).
   • Iterative Methods: VCode (using Claude Sonnet 4.5).
4. Analysis of Trade-offs: Provided a detailed analysis of the trade-off between semantic similarity and pixel-level fidelity in current models.

4. Results and Findings

The benchmarking results reveal that current technology is insufficient for reliable real-world SVG extraction:

• Performance Gap: Even the strongest models (GPT-5, Claude 4.1) plateau significantly below the thresholds required for high-fidelity generation.
  • Current Best: ~DINO 90, LPIPS 30, SSIM 60, L2 15.
  • Target for High Fidelity: ~DINO 95, LPIPS 10, SSIM 80, L2 9.
• Synthetic vs. Natural: Models perform consistently better on the Synthetic split. The Natural split exposes weaknesses in handling noise, shadows, and perspective distortion.
• Semantic vs. Fidelity Trade-off:
  • Most VLMs (e.g., GPT-5, Claude) prioritize semantic similarity over pixel fidelity. They often use text primitives to approximate shapes (e.g., rendering a letter as text rather than a vector path), leading to high DINO/LPIPS scores but poor L2/SSIM scores.
  • Qwen2.5VL-8B SVG RL and VCode take a different approach, prioritizing visual aesthetics (shape primitives) over semantic text representation. While this improves visual closeness, it sometimes sacrifices semantic accuracy or results in overly verbose code.
• Two-Step Limitations: Providing perfect bounding boxes (Two-Step) does not universally improve results across all model families. Notably, the StarVector-trained model (Qwen2.5VL-8B SVG RL) failed to utilize the crop effectively in the one-step setting, often attempting to vectorize the entire image, suggesting a misalignment in its training pipeline between text prompts and image features.
• Iterative Refinement: Methods like VCode (iterative analysis and refinement) show promise, improving performance on Claude Sonnet 4.5 by better focusing on the target and reducing background noise, though they still fall short of perfect fidelity.

5. Significance and Future Directions

• Foundation for Research: WildSVG establishes the necessary infrastructure for systematic study of SVG extraction, a task previously hindered by data scarcity.
• Real-World Applicability: The results highlight that while MLLMs are powerful, they are not yet ready for production-grade vectorization of real-world images without significant refinement.
• Future Research Directions:
  1. Alignment: Improving the alignment between focus prompts and structured vector outputs, particularly for models trained on separate text-to-SVG and image-to-SVG tasks.
  2. Integrated Training: Jointly training VLMs on both SVG generation and extraction to improve vector code fidelity in noisy contexts.
  3. Iterative Approaches: Further exploring iterative refinement pipelines to bridge the gap between semantic understanding and geometric precision.
  4. Dataset Expansion: Scaling WildSVG to support pre-training rather than just fine-tuning.

In conclusion, WildSVG identifies a critical bottleneck in the field of computer vision and graphics: the inability of current models to robustly extract vector graphics from natural scenes. The benchmark serves as a catalyst for developing next-generation models capable of handling the complexity of the "wild" world.