Notational Animating: An Interactive Approach to Creating and Editing Animation Keyframes

This paper introduces "notational animating," an interactive animation authoring paradigm that enables users to sketch high-level notations over static drawings, which are then interpreted by AI into formalized animation keyframes through a structured representation and a closed feedback loop for resolving ambiguity.

The Gist

Imagine you are an animator, but instead of spending hours manually moving a character's arm frame-by-frame in a complex software program, you can simply draw a few arrows and scribbles on a picture, and the computer magically figures out how the character should move.

This paper introduces a new way of making animations called "Notational Animating."

Here is the breakdown of the idea, using simple analogies:

1. The Problem: The "Translator" Gap

Traditionally, animators have two ways to work:

• The Old Way (Manual): Like a puppeteer pulling strings. You have to manually adjust every joint and timing. It's precise but slow.
• The New AI Way (Text Prompts): You type, "Make the frog jump happily." But AI is bad at nuance. It might make the frog jump too high, or look sad, or jump in the wrong direction. Text is too vague for the tiny details animators care about.

The Gap: Animators have a "mental image" of a cool move, but they can't easily tell the computer exactly what they mean without getting bogged down in technical settings.

2. The Solution: "Notational Animating"

The authors propose a middle ground. They realized that for decades, professional animators have used sketches to plan their work. They draw a stick figure and add a squiggly line to show "whoosh!" or a big arrow to show "heavy impact."

The Concept:
Instead of typing instructions or dragging sliders, you draw your intent directly on the static image.

• Draw an arrow? The computer knows the object should move that way.
• Draw a wavy line? The computer knows the object should wiggle.
• Draw a thick, jagged line? The computer knows the movement should be fast and violent.

Think of it like conducting an orchestra. You don't tell the violinist, "Move your bow 3 inches to the left." You wave your baton, and they know the feeling and direction of the music. This system lets you "conduct" the animation with your pen.

3. How It Works (The Magic Trick)

The system uses a smart AI (a Vision-Language Model) to act as a super-intelligent translator.

1. You Sketch: You draw a static picture of a character and add your "notation" (arrows, circles, text like "BOOM").
2. The AI Decodes: The AI looks at your drawing and asks: "Okay, this arrow points up, and the line is thick. The user wants the character to jump up with a lot of force."
3. The "Safety Net" (Feedback Loop): Because the AI might guess wrong, it doesn't just spit out the final movie immediately. It shows you a label next to your drawing saying, "I think you meant: Jump Up, High Force."
   • If you say, "No, I meant a small hop," you can just tweak the arrow or slide a little bar on the screen, and the AI updates its guess instantly.
4. The Result: The AI generates the next "keyframe" (a new picture of the character in motion). You repeat this process, building a sequence of keyframes, which the computer then smooths out into a full video.

4. Why This is a Big Deal

The researchers tested this with 7 professional animators and found some cool things:

• It feels natural: Animators didn't have to learn a new language. They just used the same "scribbles" they've used for years on paper.
• It changes how you think: Instead of thinking, "I need to move the left leg, then the right leg, then the head," animators started thinking, "I want the whole body to feel like it's bouncing." It shifts the focus from mechanics (how the parts move) to vibe (how the motion feels).
• It handles "Exaggeration": Animation is often about making things look bigger and more dramatic than real life. The system learned to understand that a giant, scribbled arrow means "super fast and huge," not just "move a little bit."

5. The Catch (Limitations)

It's not perfect yet.

• 3D is hard: Drawing a 3D twist on a flat piece of paper is confusing, even for humans. The AI sometimes gets lost if you try to describe a complex spin.
• Speed: The AI takes a few seconds to think and generate the next picture. If it were instant, animators could do even more.
• Precision: If you need pixel-perfect control (like a character's finger touching a specific button), you still need the old-school manual tools. This tool is for the "big picture" and the "feel."

The Bottom Line

Notational Animating is like giving animators a magic sketchbook. You draw your ideas as you normally would, and the computer acts as a creative partner that understands your messy, artistic scribbles and turns them into real motion. It bridges the gap between the artist's imagination and the computer's execution, making animation feel less like engineering and more like drawing.

Technical Summary

Here is a detailed technical summary of the paper "Notational Animating: An Interactive Approach to Creating and Editing Animation Keyframes."

1. Problem Statement

Traditional animation authoring tools rely on either rigid, system-defined visual abstractions (e.g., specific gestures mapping to fixed effects) or natural language prompts. Both approaches have significant limitations:

• System-defined tools enforce a fixed vocabulary that users must memorize, limiting improvisation and generalizability.
• Natural language is often too ambiguous to capture the nuanced, visual intent required for expressive animation (e.g., specific timing, weight, and dynamics).
• The "Gulf" Problem: There is a gap between an animator's mental image (gulf of envisioning) and the system's execution (gulf of execution). Furthermore, the inherent ambiguity of sketch-based notations widens the "gulf of evaluation," as users struggle to understand how the system interpreted their vague marks.

The paper addresses the need for an interaction paradigm that allows animators to use free-form, user-defined sketches (notations) to drive generative AI, while providing mechanisms to resolve ambiguity and offer fine-grained control.

2. Methodology

The research followed a design science approach, comprising three main phases:

A. Design Study & Content Analysis

• Data Collection: The authors analyzed 135 real-world animation sketches from professional animators, supplemented by interviews with two industry experts (25 and 15 years of experience).
• Notation Taxonomy: They categorized animator notations into three roles:
  1. Geometric Guides: (Focus of this work) Notations defining shape deformation, kinematics, force, path, and pose (e.g., arrows for force, curves for line of action). These are hidden from the final audience.
  2. Spatial Guides: Spanning multiple frames to define alignment and perspective.
  3. Motion Amplifiers: Visual cues (e.g., motion lines) for the audience to perceive speed/direction.
• Key Characteristics Identified: Animator notations are contextual (meaning depends on the scene), combinational (multiple marks define a single action), and ambiguous (lack pixel-level precision).

B. Formalization of Animation Intent

To bridge the gap between informal sketches and AI execution, the authors proposed a Structured Animation Representation:

• Primary Notations: Decomposed into a triplet: $\langle \text{Source}, \text{Path}, \text{Target} \rangle$.
  • Source: The object or body part moving.
  • Path: The trajectory, force, or direction (e.g., arrow, curve).
  • Target: The end state or destination.
• Secondary Notations: Stylistic descriptors (color, thickness, text labels) providing metadata like timing, intensity, or motion style.
• Flexibility: The system supports "flexible completeness," inferring missing fields (e.g., if a target is omitted) from context using Vision-Language Models (VLMs).

C. System Prototype Implementation

A prototype system was built with the following pipeline (Figure 6 in the paper):

1. Input: User draws on a static image and adds notations.
2. Grid Overlay: A 30x30 coordinate grid is overlaid to help the VLM ground spatial descriptions.
3. VLM Interpretation: An LLM (OpenAI o3) analyzes the sketch to:
   • Identify animation units.
   • Extract the structured $\langle \text{Source}, \text{Path}, \text{Target} \rangle$ triplet.
   • Generate natural language prompts for the image generator.
4. Keyframe Generation: An image generation model (Gemini-2.5-flash-image-preview) synthesizes the next keyframe based on the prompt and the previous frame.
5. Feedback Loop:
   • In-place Motion Labels: Colored tags appear near notations showing the system's interpretation (Source, Path, Target) and confidence.
   • Dynamic UI Widgets: Sliders appear for fine-tuning spatial range and intensity (e.g., "how high the hand raises").
   • Timeline: Automatically constructed to visualize per-object tracks, allowing users to adjust timing and duration.

3. Key Contributions

1. Concept of Notational Animating: A new interaction paradigm where high-level, user-defined sketches serve as the primary interface for AI-driven keyframe generation, shifting from rigid constraints to flexible intent expression.
2. Formalized Representation: A structured $\langle \text{Source}, \text{Path}, \text{Target} \rangle$ model that translates ambiguous sketches into system-executable actions without forcing a fixed vocabulary.
3. Two-Level Feedback Mechanism: A novel interface combining high-level semantic tags (to verify intent) and low-level dynamic widgets (to adjust parameters), effectively narrowing the gulf of execution and evaluation.
4. Expert Evaluation Insights: A preliminary study with 7 professional animators revealing how this paradigm shifts mental models from "doing one thing at a time" to "composing multiple elements together."

4. Results (Expert Evaluation)

A study with 7 professional animators yielded the following findings:

• Usability & Intuitiveness: Participants found the system intuitive, noting it aligned with their existing sketching workflows. They did not need to learn a new "language."
• Notating Patterns: Four distinct patterns emerged:
  1. Global Vibe: Abstract notations defining overall motion flow.
  2. Target Pose First: Explicitly drawing the desired end state.
  3. Global Dynamism + Local Highlights: Establishing overall motion, then refining specific parts.
  4. Individual Pieces First: Decomposing the scene into separate parts.
• Handling Ambiguity: When the AI misinterpreted intent, users primarily iterated on their notations (adding text labels, changing colors, or redrawing arrows) rather than editing the AI output directly.
• Mental Model Shift: Animators reported a shift from low-level parameter tweaking (e.g., X/Y coordinates) to high-level "vibe" steering. They also began thinking in terms of simultaneous composition (animating multiple interacting objects at once) rather than sequential, isolated tasks.
• Limitations Observed:
  • 3D Complexity: Difficulty in specifying complex 3D rotations (twist/tilt) on a 2D canvas.
  • Model Fidelity: Issues with spatial mismatch (left/right flipping), resistance to extreme deformation (squash/stretch), and stylistic drift.
  • Latency: High inference latency discouraged rapid iteration.

5. Significance and Future Directions

• Paradigm Shift: The paper argues for a middle-layer representation that bridges the gap between human sketching (abstract, fluid) and AI execution (concrete, pixel-based). This preserves the animator's agency while leveraging generative AI for tedious tasks.
• Production Potential: While currently an ideation tool, the authors envision it evolving into a production-ready system as video generation models improve in speed and fidelity.
• Future Work:
  • Integrating camera movement and non-geometric transformations (lighting, color).
  • Developing mechanisms to dynamically regulate AI "creativity" based on the user's level of abstraction (e.g., inhibiting style changes when the user provides precise constraints).
  • Longitudinal studies to assess the impact on real-world production pipelines.

In summary, Notational Animating offers a promising path toward AI-assisted animation that respects the intuitive, visual language of artists, moving beyond rigid toolsets and ambiguous text prompts to a collaborative, sketch-based workflow.