Technical Summary: Stroke of Surprise

Problem Definition

The paper introduces Progressive Semantic Illusions, a novel task in vector sketching that extends visual illusions from the spatial dimension to the temporal dimension. Unlike traditional visual illusions (e.g., visual anagrams, shadow art) that rely on spatial manipulations like viewpoint changes or frequency band interleaving, this task requires a single vector sketch to undergo a dramatic semantic transformation through the sequential accumulation of strokes.

The core challenge is the "Dual-Constraint":

1. Prefix Constraint: The initial set of strokes ($S_{prefix}$) must form a coherent, recognizable object (Concept A).
2. Delta Constraint: The subsequent addition of strokes ($S_{delta}$) must re-contextualize the entire composition into a distinctly different object (Concept B), without merely overwriting or occluding the initial structure.

Existing methods fail to address this additive nature. Raster-based approaches (e.g., Nano Banana Pro) rely on destructive editing, violating the progressive constraint by overwriting pixels. Sequential vector methods (e.g., SketchAgent, SketchDreamer) employ a greedy strategy, optimizing prefix strokes solely for Concept A. This results in the prefix becoming "semantic noise" or clutter when Concept B is introduced, as the initial strokes are "frozen" in a local minimum incompatible with the second concept. The paper argues that a "Common Structural Subspace"—a shared geometric configuration valid for both semantic interpretations—must be discovered through joint optimization.

Methodology

The authors propose Stroke of Surprise, a generative framework that optimizes vector strokes to satisfy distinct semantic interpretations at different drawing stages. The method treats the problem as a constrained optimization over shared Bézier curve parameters.

1. Joint Optimization Pipeline

Instead of sequential generation, the framework employs a dual-branch Score Distillation Sampling (SDS) mechanism to optimize all stroke parameters ($\theta$) simultaneously:

• Prefix Branch: Renders only the prefix strokes ($S_{prefix}$) and guides them using a frozen diffusion model conditioned on the prompt for Concept A ($p_1$). This generates the prefix SDS loss ($\mathcal{L}_{SDS}^{prefix}$).
• Full Branch: Renders the complete set of strokes ($S_{full} = S_{prefix} \cup S_{delta}$) and guides them using the diffusion model conditioned on the prompt for Concept B ($p_2$). This generates the full SDS loss ($\mathcal{L}_{SDS}^{full}$).
• Total Loss: The gradients from both branches are combined ($\mathcal{L}_{SDS} = \mathcal{L}_{SDS}^{prefix} + \mathcal{L}_{SDS}^{full}$) and backpropagated to update all stroke parameters. This ensures prefix strokes are "primed" to serve dual roles: recognizable as Concept A while structurally supporting Concept B.

2. Overlay Loss for Spatial Coordination

To prevent the delta strokes from simply cluttering or occluding the prefix (a common failure mode of pure semantic guidance), the authors introduce a novel Overlay Loss ($\mathcal{L}_{overlay}$).

• Mechanism: The prefix and delta stroke maps are rendered separately and blurred with a Gaussian kernel to create soft spatial buffers.
• Objective: The loss computes the normalized inner product of these blurred maps, penalizing spatial overlap.
• Effect: This enforces spatial complementarity, ensuring that delta strokes structurally integrate with and re-contextualize the prefix (e.g., transforming pig ears into angel wings) rather than obscuring them.

3. Evaluation and Ranking

The framework includes a VLM-based filtering and ranking pipeline using GPT-4o to select high-quality candidates. The evaluation assesses:

• Phase Recognizability: How clearly the prefix depicts Concept A and the full sketch depicts Concept B.
• Structural Integrity: Whether the full sketch is significantly more recognizable than the delta strokes alone, ensuring the prefix provides essential scaffolding.
• Ranking Metrics: A composite score combining CLIP, ImageReward, and Human Preference Score (HPS) metrics, weighted to penalize cases where the delta strokes alone could satisfy the target (indicating the prefix was not structurally integrated).

4. Extensions

The framework scales to $K$-phase illusions (e.g., A $\to$ B $\to$ C) by partitioning strokes into disjoint subsets and employing parallel branches for each cumulative stage. It also generalizes to B-splines, colored strokes, and general vector graphics.

Key Contributions

1. Task Definition: The introduction of Progressive Semantic Illusions, shifting visual illusions from spatial to temporal dimensions via additive stroke accumulation.
2. Method: A sequence-aware joint optimization framework that discovers a "Common Structural Subspace" using dual-branch SDS.
3. Novel Loss: The introduction of Overlay Loss to enforce spatial complementarity and structural integration, preventing occlusion and clutter.
4. Evaluation: A comprehensive evaluation pipeline including quantitative metrics (structural concealment, semantic concealment) and user studies demonstrating superior performance over state-of-the-art baselines.

Results

Extensive experiments on a dataset of 64 object pairs demonstrate that Stroke of Surprise significantly outperforms baselines (CLIPasso, ControlSketch, SketchDreamer, SketchAgent, Nano Banana Pro):

• Recognizability & Coverage: The method achieves 100% coverage of successful illusions, compared to 34.9% for Nano Banana Pro and lower scores for vector baselines.
• Concealment Scores: It achieves the highest structural concealment scores, indicating that prefix strokes retain significant structural roles in the final concept rather than being overwritten.
• User Studies: In a study with 143 participants, the method was preferred in 67.7% to 87.1% of cases over baselines, depending on the ranking strategy used.
• Ablation Studies:
  • Joint vs. Sequential: Joint optimization is essential; sequential approaches fail to find the common subspace, resulting in rigid prefixes that conflict with the second concept.
  • Overlay Loss: Removing this loss leads to significant degradation in structural concealment, confirming its necessity for clean integration.
  • Initialization: Centered, gathered initialization is critical for convergence in this non-convex optimization landscape.

Significance

The paper claims that Stroke of Surprise is the first framework to enable real-time semantic re-contextualization in vector sketching. By shifting from spatial to temporal dimensions, it allows a single vector asset to reveal distinct semantic interpretations through progressive stroke accumulation. The work demonstrates that prefix strokes must be "primed" for future semantics through joint optimization, a capability greedy baselines lack. The framework successfully expands visual anagrams from the spatial to the temporal dimension, offering a new paradigm for dynamic visual storytelling, brand identity animation, and cognitive science research into temporal semantic perception.