💻 computer science

CART Rule-Guided Diffusion and AI-Synthesized User Preferences for New Media Art Generation

This paper proposes a CART-rule-guided diffusion model that leverages a conditional GAN to synthesize large-scale user preference data and integrates interpretable decision-tree rules into the diffusion sampling process, significantly enhancing the aesthetic quality and personalization of new media art generation while overcoming the ethical and cost barriers of human data collection.

Original authors: Caixian Ye, Lijun Xu, Jun Tang

Published 2026-10-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Caixian Ye, Lijun Xu, Jun Tang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Art has always been a conversation between the creator and the viewer, a dialogue where meaning is shared and emotions are felt. In recent years, artificial intelligence has joined this conversation, capable of painting images that look as real as photographs or as stylized as a dream. Yet, a quiet frustration remains. While these machines can produce technically perfect pictures, they often struggle to understand what a specific person finds beautiful. They generate content that is competent but emotionally flat, offering a menu of options from which a user must passively choose, rather than anticipating what that user truly wants to see. This gap between technical skill and personal taste is the central challenge researchers are now trying to solve.

A team of researchers at Guangzhou Xinhua University has proposed a new way to bridge this divide, moving beyond simple selection to active, personalized creation. Their work focuses on a specific type of artificial intelligence known as a diffusion model, which builds images by gradually removing noise from a chaotic cloud of pixels until a clear picture emerges. The innovation lies in how they guide this process. Instead of letting the machine wander freely, they teach it to follow a set of interpretable rules derived from human taste. To do this, they first had to solve a major hurdle: gathering enough human opinions to train the system without the immense cost and ethical complications of surveying thousands of real people.

The researchers began with a small seed of real data, consisting of just two thousand images that had been rated by human experts on various artistic qualities like color harmony and composition. From this modest starting point, they used a secondary artificial intelligence to create a virtual population of ten thousand simulated users. These digital agents generated half a million preference records, effectively creating a massive library of taste that mirrored the real human seed data. The team verified that this synthetic library was statistically indistinguishable from the real thing, ensuring that the machine was learning from a faithful representation of human preference rather than a distorted imitation.

With this vast library of preferences in hand, the researchers trained a decision tree, a type of algorithm that sorts information through a series of simple yes-or-no questions, much like a flowchart. This tree learned to identify the specific combinations of features that led to high aesthetic scores. Crucially, the team did not just use these rules to filter out bad images after they were made. Instead, they converted the rules into a gentle, continuous signal that was injected directly into the image-making process. As the machine built an image step by step, this signal acted as a subtle nudge, steering the generation toward the specific style and quality that the virtual users preferred. It was a shift from asking the machine to guess what you like, to teaching it how to feel your taste as it works.

To ensure the system kept improving, the researchers built a closed-loop feedback mechanism. After the machine generated a set of images, the virtual users scored them, and the system used these scores to slightly adjust the importance of the rules it was following. This allowed the model to refine its own understanding over time, converging on a strategy that produced higher-quality, more personalized art with each iteration. The results were significant. When compared to other leading methods, this new approach produced images that were not only closer to the desired style but also of higher overall quality. The system reduced the distance between the generated art and the ideal aesthetic by nearly fourteen percent and improved the accuracy of matching user preferences by nearly ten percent.

The study also revealed that each part of their system was essential. Removing the decision tree caused the quality of the images to drop noticeably, proving that the rules provided necessary guidance. Removing the synthetic data library forced the system to train much longer and with less success, showing the value of the expanded virtual population. And removing the feedback loop meant the system could not refine its approach, highlighting the importance of continuous adjustment. The researchers found that their method was also far more efficient, training in a fraction of the time required by other complex models.

While the results are promising, the authors are careful to note that these achievements were measured within a simulated environment. The virtual users, while statistically faithful, cannot fully capture the complex, emotional, and cultural nuances of real human art appreciation. The team has outlined a plan to test their system with real people in a controlled setting and later through a large online study to confirm that the improvements hold up in the real world. Until then, this work stands as a powerful demonstration that by combining interpretable rules with synthetic data, it is possible to teach machines to create art that feels less like a machine output and more like a personal expression.

Technical Summary: CART Rule-Guided Diffusion and AI-Synthesized User Preferences for New Media Art Generation

1. Problem Statement

While AI-generated content (AIGC) and diffusion models have achieved near-photographic quality, current pipelines remain largely "behavior-centric," producing homogeneous outputs that lack deep alignment with individual aesthetic preferences. The paper identifies two primary structural obstacles to preference-driven AIGC:

Data Scarcity and Ethical Friction: Training personalized models typically requires large-scale human-subject data, which is costly to collect, difficult to scale, and fraught with ethical considerations regarding privacy and bias.
Lack of Interpretable Guidance: Existing methods often treat user preferences as post-hoc filters or rely on black-box predictors. There is a lack of mechanisms to inject interpretable, rule-based guidance directly into the diffusion sampling trajectory in real-time.

2. Methodology

The authors propose a framework integrating three core components: a CGAN-based virtual user pool, a CART-rule-guided diffusion model, and a closed-loop feedback mechanism.

A. AI-Synthesized User Preferences (CGAN)

To overcome data scarcity, the authors construct a Conditional Generative Adversarial Network (CGAN) to synthesize a virtual user pool.

Seed Data: A small, real-user seed dataset ( $N=2,000$ ) is assembled from WikiArt and ArtBench-10, annotated by three experts on five aesthetic dimensions (color harmony, composition novelty, etc.) across 12 distinct new media art styles.
Synthesis: The CGAN is trained to map a condition vector (style label, age encoding, preference type) and noise to a preference vector.
Scale: This process generates 10,000 virtual user agents, each rating 50 random styles, resulting in $5 \times 10^5$ preference records.
Validation: The synthetic data is validated against the seed data using Kullback–Leibler divergence ( $D_{KL} < 0.05$ ), Maximum Mean Discrepancy (MMD $< 0.01$ ), and Fréchet Distance (FD deviation $< 10\%$ ), confirming statistical fidelity.

B. CART Rule-Guided Diffusion

The framework converts preference modeling into a differentiable guidance signal for the diffusion process.

Rule Extraction: A CART (Classification and Regression Tree) decision tree is trained on the synthetic preference data. The tree partitions the feature space into "if–then" rules anchored at leaf nodes.
Differentiable Injection: Since standard CART rules are discrete and non-differentiable, the authors define a "soft violation measure" using a sigmoid function to create a continuous, differentiable distance metric ( $d_r(x)$ ).
Gradient Signal: These rules are converted into a composite constraint function $C(x; R)$ . The gradient of the log-constraint, $\nabla_{x_t} \log C(x_t; R)$ , is injected into the reverse sampling trajectory of the Stable Diffusion 1.5 backbone at every $K$ -th denoising step.
Formula: The update step is modified as:
$x_{t-1} = \mu(x_t) + \sigma_t \cdot \epsilon + \lambda \cdot \nabla_{x_t} \log C(x_t; R)$
where $\lambda$ is the rule guidance strength.

C. Closed-Loop Feedback Mechanism

A coordinator iteratively updates the system to refine rule weights without retraining the entire tree structure.

Process: The system executes a cycle of generation $\rightarrow$ agent scoring $\rightarrow$ deviation feedback $\rightarrow$ rule update.
Optimization: The coordinator freezes the CART structure but incrementally adjusts leaf-node prediction weights and confidence values based on scoring deviations.
Termination: The loop terminates when the validation Fréchet Inception Distance (FID) fails to improve for five consecutive rounds.

3. Key Contributions

The paper outlines three coordinated contributions:

Scalable Preference Synthesis: Replacing the human-subject bottleneck with a CGAN-based virtual user generator that scales a 2,000-sample seed into 500,000 synthetic records while maintaining distributional fidelity.
Differentiable Rule Guidance: Elevating CART rules from discrete, offline filters to a differentiable gradient signal injected directly into the reverse diffusion process, enabling online dynamic guidance.
Co-Evolutionary Feedback: Implementing a closed-loop coordinator that allows the rule set to co-evolve with generated artifacts through iterative weight updates, rather than static post-screening.

4. Experimental Results

The proposed model was evaluated against five baselines (AIGC-only, CF-AIGC, MLP-AIGC, RF-AIGC, and DDTs-AIGC) and three ablation variants.

Performance Metrics: The proposed model achieved:
- FID: 29.56 (13.8% improvement over the strongest baseline, RF-AIGC).
- Preference Accuracy (PA): 0.734 (9.4% improvement).
- Hit Rate@3: 0.512 (19.6% improvement).
- Efficiency: Training time was reduced by 78% compared to MLP and 74% compared to Random Forest.
Ablation Studies:
- Removing the CART module increased FID by 14.5% and reduced PA by 15.8%, confirming the necessity of rule-based compression.
- Removing CGAN (training only on real data) doubled the convergence epochs (from 87 to 189) and increased FID by 29.3%, highlighting the role of synthetic data in expanding aesthetic coverage.
- Removing the Closed-Loop resulted in a smaller but measurable degradation, indicating the loop provides incremental optimization.
Convergence: The closed-loop system demonstrated monotonic improvement in FID, stabilizing after approximately five iterations.

5. Significance and Claims

The paper claims that integrating interpretable rule-based guidance with AI-synthesized preference data significantly enhances both the aesthetic quality and personalization of AIGC outputs.

Theoretical Insight: The success of CART over neural networks (MLP/RF) suggests that binary recursive partitioning is structurally isomorphic to the hierarchical nature of human aesthetic judgment (perception $\rightarrow$ classification $\rightarrow$ evaluation), offering piecewise-linear decision boundaries that better match non-convex preference distributions.
Practical Impact: The framework offers a reproducible, low-cost alternative to human-rating pipelines, lowering the entry barrier for small studios. The lightweight nature of the rule guidance (latency of ~4.3s) supports real-time new media art installations.
Modesty and Limitations: The authors explicitly acknowledge that current results are based on a virtual user pool and have not yet been validated against real human subjects. They note that synthetic data cannot encode higher-order cognitive factors like emotional resonance or cultural context. Consequently, the reported gains quantify performance relative to the synthetic distribution, and the authors propose a pre-registered two-stage study (in-lab and cross-cultural online) to validate ecological validity before real-world deployment.