💻 computer science

Texture Regenerating and Grafting Using Genome-Driven Neural Cellular Automata

This paper introduces a novel training methodology and inference-based grafting technique for Genome-Driven Neural Cellular Automata that enable robust self-regeneration of damaged textures and the seamless combination of distinct textures without requiring retraining.

Original authors: Mirela-Magdalena Catrina, Ioana Cristina Plajer, Alexandra Băicoianu

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mirela-Magdalena Catrina, Ioana Cristina Plajer, Alexandra Băicoianu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical, living wallpaper that can fix itself if you tear it, and you can also stick two different wallpapers together so they grow into one seamless piece. That is essentially what this paper is about, but instead of paper, it uses a computer program called a Neural Cellular Automaton (NCA).

Here is a simple breakdown of what the researchers did, using everyday analogies.

1. The Magic Wallpaper (The NCA)

Think of the NCA as a giant grid of tiny pixels. Each pixel is like a little robot that only looks at its immediate neighbors (the pixels right next to it) to decide what color it should be.

How it works: Instead of being told exactly what to draw, these little robots follow a set of hidden rules (a "genome"). If you give them a starting pattern, they all talk to their neighbors and slowly evolve until the whole grid looks like a specific texture, like wood grain, clouds, or fabric.
The Innovation: Usually, you need a different set of rules for every single texture. The researchers found a way to pack many different rulebooks (genomes) into one single program. It's like having one master chef who can cook Italian, Chinese, and Mexican food just by reading a different menu card (genome) at the start.

2. Healing the Torn Fabric (Texture Regeneration)

Imagine you have a beautiful piece of this living wallpaper, but someone spills coffee on a patch of it, or you tear a hole in the middle.

The Problem: In the past, if you damaged a texture, the computer often got confused. It might try to fix the hole by growing the wrong pattern (like trying to fix a wood grain pattern with a cloud pattern).
The Solution: The researchers taught the program a new trick during its training. They deliberately "scuffed up" the training data—hiding parts of the textures with noise or holes—and told the program, "Fix this!"
The Result: Now, if you damage the wallpaper, the little robots look at the healthy neighbors around the hole and say, "Ah, I see what's missing," and they grow the correct pattern back into the empty space. It's like a biological organism healing a wound.

3. Grafting: Stitching Two Worlds Together

This is the most creative part. "Grafting" is a term from gardening where you take a branch from one plant and attach it to another so they grow together.

The Experiment: The researchers took two different textures (say, a brick wall and a grassy field) and told the program to start with the "brick" rules on the left side and the "grass" rules on the right side.
The Magic: They didn't just paste them together with a hard line. They created a "fuzzy" border where the rules mixed. As the program evolved, the brick and the grass didn't just sit next to each other; they grew into each other, creating a smooth, natural transition zone where the two textures blend.
No Retraining Needed: The best part is that they didn't have to teach the program how to do this. The program already knew the rules for both textures, so it could figure out how to merge them on the fly.

4. The Catch (Limitations)

The paper is honest about a few quirks:

The Drift: Because the textures are "alive" and constantly evolving, if you let them run for too long, one texture might slowly start to eat into the other. It's like two crowds of people slowly merging; eventually, one group might take over the whole room.
Size Matters: If the hole you are trying to fix is too big, the little robots get confused because they can't see far enough to know what the pattern should be. It works great for small scratches but struggles with massive holes.
Not a Photocopier: The program captures the vibe or style of the texture (like the feel of sand), but it doesn't copy the exact pixels of the original image. It creates a new, similar version.

Summary

In short, the researchers built a smart, self-healing digital fabric. They taught it how to fix its own holes and how to seamlessly merge two different patterns together without needing a new teacher for every new job. It's a lightweight, efficient way to create complex, living textures that can repair themselves and adapt to new shapes.

Technical Summary: Texture Regenerating and Grafting Using Genome-Driven Neural Cellular Automata

Problem Statement
Texture synthesis is a computationally demanding task in computer graphics, traditionally addressed by methods such as Markov random fields, wavelet decompositions, and modern deep learning models like CNNs, GANs, and Transformers. While these generative models achieve high visual fidelity, they often incur significant computational costs and require distinct architectures for specific tasks. Furthermore, specialized domains such as texture regeneration (inpainting damaged regions) and texture grafting (seamlessly combining distinct textures) remain under-explored. Existing Neural Cellular Automata (NCA) approaches, while efficient, typically require training a separate automaton for each texture, which hinders seamless interpolation and complicates grafting operations that often rely on explicit stitching.

Methodology
The authors propose an extension to genome-driven NCAs, building upon the framework introduced in [13]. The core architecture defines an NCA on a 2D grid where each cell is an $n$ -dimensional vector. The first three channels represent RGB color, while the remaining channels include "genome" channels and hidden channels. The genome channels encode a binary representation of the target texture index, allowing a single NCA to generate multiple distinct textures ( $2^{n_g}$ textures for $n_g$ genome channels).

The methodology introduces two primary innovations:

Regeneration-Adapted Training: To enable robust self-regeneration of damaged textures within a multi-texture setting, the authors modify the standard pooling training strategy. In the proposed Algorithm 1, the training pool is initialized with seeds for all target textures. During each epoch, the algorithm replaces the highest-scoring (lowest loss) state of a specific genome with its seed state to prevent degradation. Crucially, to teach the NCA to repair damage, the two lowest-scoring states in the batch are "damaged" by randomizing cell states within a circular region (radius 15–25 pixels). The NCA then evolves to reconstruct the original texture from this corrupted state. This process is repeated over multiple evolution steps before computing loss via a pretrained VGG discriminator and Sliced Wasserstein Loss (SWL).
Genome-Based Grafting: The paper introduces a grafting technique that operates during the inference phase without retraining. By initializing a single NCA grid with different genome values assigned to specific spatial regions (e.g., left vs. right halves, or concentric circles), multiple textures are forced to coexist and interact. To ensure smooth transitions at the boundaries, the genome channels of cells at the interface are initialized with intermediate values (interpolating between the binary genome codes). The NCA then evolves iteratively, allowing the textures to grow together and form a unified, coherent surface.

Key Contributions

Robust Multi-Texture Regeneration: The study demonstrates that a single NCA, trained with the adapted pooling strategy, can successfully regenerate damaged regions for any texture in its training set. This extends the self-organizing properties of NCAs beyond simple synthesis to dynamic self-repair.
Seamless Grafting without Retraining: The authors present a versatile grafting technique that combines distinct textures into a novel pattern during inference. This is achieved by manipulating the initial genome channels of the NCA cells, eliminating the need for specialized retraining or complex stitching algorithms.
Efficient Paradigm: The approach leverages the inherent local pattern generation and self-organization of NCAs, offering a computationally lighter alternative to large-scale generative models while maintaining the ability to handle complex texture compositions.

Results

Quantitative Performance: Evaluation using Structural Similarity (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and Gram Matrix Distance (GMD) shows that the regeneration-adapted NCA performs competitively with the baseline NCA from [13], indicating that the adaptation for regeneration does not degrade generative quality.
Regeneration Capabilities: The model successfully reconstructs textures with damage radii up to 25 pixels. While global patterns are restored, the authors note that fine-grained details may converge to alternative local configurations for larger damages (35+ pixels), leading to lower local metric scores despite perceptual seamlessness.
Grafting Outcomes: Experiments demonstrate the successful combination of 2 and 3 distinct textures. The NCA generates consistent boundary transitions, ranging from smooth gradients to defined barriers. However, the results also highlight "genomic competition," where, over long evolution times, more stable textures may encroach upon less stable ones, causing boundary drift. This instability is mitigated by increasing the texture dimension.

Significance and Claims
The paper claims that this work significantly advances multi-texture synthesis by introducing a training methodology that enables robust self-regeneration and a grafting technique that allows for seamless texture combination without retraining. The authors position this as a powerful and efficient paradigm for dynamic texture composition and self-repair in autonomous systems.

The significance is framed not only in computer graphics but also in broader fields exploring self-organizing systems (physics, robotics, biology). The work highlights the fundamental properties of NCAs, demonstrating that a single, lightweight architecture can handle diverse textures, repair damage, and perform complex grafting operations. The authors modestly acknowledge limitations, noting that the model captures stylistic essence rather than pixel-perfect reproduction and that genomic competition can lead to instability in long-term simulations. They suggest that future work should investigate hierarchical architectures to improve fidelity and theoretical analysis of boundary dynamics to better understand the stability of multi-class self-organizing systems.