💻 computer science

Interactive Stroke-based Neural SDF Sculpting

This paper introduces an interactive framework that enables users to perform real-time, stroke-based sculpting directly on neural implicit SDF representations, overcoming previous limitations of slow optimization and spot-only edits to provide intuitive, high-performance 3D shape manipulation.

Original authors: Fizza Rubab, Yiying Tong

Published 2026-08-20

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Fizza Rubab, Yiying Tong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of three-dimensional design, artists have long relied on digital meshes, which are essentially complex webs of tiny triangles that form the skin of a virtual object. These meshes are powerful because an artist can grab a specific point on the surface and pull it, creating a smooth, predictable change. However, a newer way of describing 3D shapes has emerged, one that does not use a mesh of triangles at all. Instead, these shapes are defined by a continuous mathematical field, a sort of invisible cloud of data that tells a computer how far any point in space is from the surface of the object. This method, known as a neural signed distance function, allows for incredibly smooth and detailed shapes that can be zoomed into without ever losing quality. The problem is that while these shapes are beautiful and efficient to store, they are notoriously difficult to change. Because the shape is hidden inside the complex weights of a computer program rather than in visible points, trying to carve or mold it has been like trying to sculpt a cloud by adjusting the weather patterns that created it.

Researchers at Michigan State University have developed a new tool that finally allows artists to sculpt these invisible, neural shapes with the same intuitive ease as traditional clay or digital meshes. Their work introduces a system where a user can draw a continuous line, or stroke, across a 3D model, and the computer instantly reshapes the surface along that path. Unlike previous attempts that required the user to click individual points and wait for slow calculations, this new method works in real time. The system allows the artist to define a custom brush shape, deciding exactly how wide or narrow the tool is, and how the pressure of the stroke changes from the beginning to the end of the line. This means an artist can carve a deep groove that fades out at the edges, or build up a ridge that is strongest in the middle, all with a single, fluid motion.

The secret to this speed lies in how the computer samples the data. Instead of trying to update the entire mathematical model every time a stroke is made, the system focuses only on a narrow, tube-shaped region surrounding the user's brush stroke. It gathers a specific set of points within this tube, calculates the necessary changes, and then quickly adjusts the underlying neural network to match the new shape. This targeted approach is vastly more efficient than older methods that tried to process the whole surface at once. In their tests, the researchers found that this technique could be up to fifteen times faster than previous point-based editing tools, allowing for a responsive experience where the shape changes immediately as the mouse moves.

The results of this work are not just about speed, but about the quality of the art that can be produced. The researchers demonstrated the tool by creating a variety of complex edits, such as adding realistic collarbones to a digital bust, carving intricate patterns into a pumpkin, or even morphing a frog into a lizard by adding scales along a wavy path. They showed that the system could handle both pushing material out to create bumps and pulling it in to create dents, all while maintaining the smooth, high-quality surface that makes neural representations so valuable. By proving that these complex, invisible shapes can be manipulated with the same freedom as traditional models, the researchers have bridged a significant gap in digital design, making a powerful new technology accessible to human creativity.

Technical Summary: Interactive Stroke-based Neural SDF Sculpting

Problem Statement

Implicit Neural Representations (INRs), particularly Neural Signed Distance Functions (SDFs), have emerged as powerful tools for modeling 3D geometry due to their ability to represent continuous surfaces at arbitrary resolution. However, a significant bottleneck remains in their editability. Unlike traditional mesh-based representations where vertices can be directly manipulated, neural SDF parameters are embedded globally within network weights. This global dependency makes local modifications challenging.

Existing approaches to editing neural SDFs often rely on modifying shape codes or latent spaces, which frequently lack fine-grained control and immediate feedback. While point-based editing methods exist (e.g., 3DNS), they are limited to localized spot edits. Attempting to simulate stroke-based sculpting by sequentially applying point edits is computationally expensive, leading to significant latency, redundant evaluations, and uneven sampling densities. Consequently, a high-performance, interactive toolkit for stroke-based sculpting on neural SDFs has been lacking, hindering the adoption of these representations in artistic and design workflows that require intuitive, real-time manipulation.

Methodology

The authors propose an interactive framework that enables stroke-based sculpting directly on pretrained neural SDFs. The system operates by sampling points along a user-defined stroke within a tubular neighborhood and fine-tuning the neural network to align its zero-level set with the desired deformation.

1. Coordinate System and Brush Definition

To facilitate intuitive control, the method establishes a local u-v-n coordinate frame along the stroke path:

u (Stroke Direction): The tangent of the stroke curve.
v (Brush Profile Direction): Perpendicular to the stroke and tangential to the surface.
n (Normal Direction): Orthogonal to both, pointing outward from the surface.

The brush is defined as a $C^1$ function, $b_P(v)$ , operating along the $v$ -direction. Users can define custom profiles using control points interpolated via Catmull-Rom splines or select analytical functions (linear, cubic, quintic). The brush intensity and radius are parameterized independently, allowing for both carving (negative displacement) and extrusion (positive displacement).

2. Modulation and Stroke Representation

A stroke is represented as a continuous curve $\gamma(u)$ parameterized by $u \in [0,1]$ . The deformation intensity can vary dynamically along the stroke via a modulation function $m(u)$ . This allows for effects such as linear falloff, central peaks, or sinusoidal patterns (e.g., scales) without additional computational cost, as the modulation is applied directly to the stroke parameterization.

3. Tubular Sampling Strategy

A core technical contribution is the tubular sampling strategy. Instead of sampling the entire surface or relying on discrete point projections, the system generates samples within a tubular region surrounding the stroke:

Points are sampled uniformly along the stroke trajectory ( $u$ -direction).
For each point, offsets are applied in the $v$ -direction within a radius $r$ .
These offset points are projected onto the surface to capture the tubular neighborhood.

This approach ensures consistent brush effects across the interaction region. The authors note that this strategy requires significantly fewer samples (e.g., ~10,000 total for stabilization) compared to point-based methods like 3DNS, which may require over 100,000 samples to achieve similar coverage, thereby drastically reducing optimization batch sizes.

4. Optimization and Loss Function

The editing process involves fine-tuning the neural SDF weights ( $\theta$ ) to minimize a deformation loss. The target positions $p'$ are computed by offsetting surface points $p$ along the normal direction $n$ based on the brush profile and modulation:
$p' = p + s \cdot b_P(v(p)) \cdot m(u) \cdot n(u(p))$

The network is optimized to ensure the new zero-level set passes through these displaced points. The total loss function combines several terms adopted from the 3DNS framework:

Regression Loss ( $L_{regression}$ ): Minimizes the SDF value at displaced points.
Eikonal Loss ( $L_{eikonal}$ ): Enforces the gradient norm to be 1, ensuring the SDF property is maintained.
Normal Loss ( $L_{normal}$ ): Aligns predicted gradients with surface normals.
Boundary Regularization ( $L_{boundary}$ ): Prevents artifacts near the bounding box edges.

Key Contributions

Interactive Stroke-Based Toolkit: The introduction of a framework that allows users to perform stroke-based modifications on neural SDFs in real-time, moving beyond the limitations of point-based editing.
Customizable Brush and Modulation: A flexible system supporting user-defined brush profiles (via control points) and modulation functions along the stroke path, enabling complex effects like chiseled strokes, asymmetric shapes, and rhythmic patterns.
Efficient Tubular Sampling: A performance-oriented sampling strategy that focuses on the interaction region, achieving up to a 15× speedup over prior point-based methods (3DNS) while maintaining edit fidelity.
Unified Representation: The ability to represent and edit complex shapes within the same neural implicit framework, eliminating the need to switch between mesh and neural representations.

Experimental Results

The method was evaluated on a dataset of six 3D shapes (including the Stanford Bunny, a bust, and a pumpkin) using an NVIDIA GeForce RTX 4070 GPU.

Speed and Interactivity: The proposed method achieves editing times under one second for various stroke densities. Table 1 in the paper demonstrates a speedup ranging from 1.18× to 15.01× compared to 3DNS as the number of stroke points increases. This efficiency is attributed to the reduced number of samples required in the interaction region.
Geometric Accuracy: Using Chamfer Distance as the metric, the method outperforms both 3DNS point-based edits and direct editing on coarse meshes. The mean Chamfer distance for the entire surface was consistently lower for the proposed method across all tested shapes (e.g., 7.279 vs. 7.394 for the sphere).
Visual Quality: The system produces smooth, continuous deformations. While high-frequency details depend on the underlying MLP capacity (SIREN), the method successfully handles complex edits such as adding robotic features to a bunny or carving intricate patterns on a pumpkin, which are difficult or impossible to achieve with overlapping point edits in 3DNS.

Significance and Claims

The paper positions this work as a bridge between traditional 3D sculpting workflows and modern neural implicit representations. The authors claim that their framework addresses the critical lack of interactive editability in neural SDFs. By enabling intuitive, stroke-based manipulation without compromising the smooth nature of implicit representations or interactive performance, the method expands the applicability of neural SDFs in artistic and design domains.

The authors modestly note limitations, particularly regarding the preservation of high-frequency details with highly oscillatory brush profiles (which may blur depending on the MLP depth) and the computational overhead of Python-based PyTorch implementations. However, they assert that for typical sculpting workflows involving low-to-medium intensity edits, the framework provides a versatile and responsive tool for 3D modeling.