🤖 AI

Culture-inspired Multi-modal Color Palette Generation and Colorization: A Chinese Youth Subculture Case

This paper addresses the lack of cultural consideration in existing color generation research by constructing a unique Chinese Youth Subculture (CYS) dataset and developing an interactive, multi-modal framework to generate culturally specific color palettes and perform automatic colorization, which is validated through a human-in-the-loop demo system and user studies.

Original authors: Yufan Li, Jinggang Zhuo, Ling Fan, Harry Jiannan Wang

Published 2026-05-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yufan Li, Jinggang Zhuo, Ling Fan, Harry Jiannan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a black-and-white sketch of a city street. If you ask a traditional artist to color it, they might use soft, natural tones. But if you ask a member of a specific, trendy youth group to color that same street, they might slap neon green on the buildings and hot pink on the sky. To them, that clash of colors isn't "ugly"; it's cool, rebellious, and full of meaning.

This paper is about teaching a computer to understand that specific "vibe." The researchers, working with Chinese Youth Subculture (CYS), built a system that doesn't just pick colors based on rules; it picks them based on culture.

Here is a breakdown of their work using simple analogies:

1. The Problem: The Computer is "Culturally Tone-Deaf"

Think of existing color tools (like the ones in Photoshop or design apps) as a chef who only knows how to cook one type of food: "Standard International Cuisine." If you ask them to make a spicy Sichuan dish, they might just add a little salt because they don't understand the concept of "spicy."

The paper argues that current AI color tools are like this chef. They ignore culture. For example, in traditional Chinese theory, mixing red and green is a fashion disaster. But for Chinese youth subculture, that exact red-and-green combo is a symbol of rebellion and coolness. The old AI doesn't get this joke; the new one needs to.

2. The Solution: A "Cultural Dictionary" (The Dataset)

To teach the AI this new language, the researchers couldn't just feed it random pictures. They had to build a special library, or a "Cultural Dictionary."

The Collection: They went online to websites popular with Chinese Gen Z (think punk, hip-hop, and techno scenes) and hand-picked 1,263 images that truly felt like that subculture.
The Ingredients: For every image, they didn't just save the picture. They saved:
- The 5 main colors (the palette).
- The text describing the image (like "evil" or "sun fun").
- The category (like "punk" or "hip-hop").
The Human Touch: They didn't just let a computer pick the top 5 colors. Real designers looked at the images and picked the 5 colors that felt right for that specific subculture, even if those colors weren't the most common ones in the picture. This ensured the "soul" of the culture was captured, not just the math.

3. The Engine: A Two-Step "Color Chef"

The researchers built a machine learning system that works like a two-person kitchen team:

Chef A (The Palette Generator): This chef looks at your request (a black-and-white photo, a text description, and a category like "techno") and says, "Okay, for a techno vibe with the word 'chaos,' here are the 5 colors we need." It uses a "conditional" brain, meaning it listens to all your hints before picking a color.
Chef B (The Colorizer): Once Chef A hands over the 5 colors, Chef B takes the black-and-white photo and paints it using only those colors, making sure the final image looks like it belongs in that specific subculture.

4. The "Human-in-the-Loop" (The Taste Tester)

The system isn't just a one-way street. The researchers built a demo where humans can play with the results.

You give the system a prompt.
It gives you a colored image.
You can tweak the colors if you don't like them.
The system remembers your tweak. It's like a student who learns from your corrections: "Oh, the user didn't like that shade of blue; next time, I'll pick a darker one." This helps the AI get smarter over time.

5. Did It Work? (The Taste Test)

To prove their system actually understood the culture, they ran a test:

They took 20 neutral keywords and asked a standard tool to make a color palette.
They asked their new "Cultural AI" to make a palette for the same keywords.
They showed these to four professional designers and asked, "Which one feels more like Chinese Youth Subculture?"

The Result: The designers picked the AI's culture-inspired choices 75% of the time for the palettes and 76% of the time for the final colored images. The AI successfully learned the "secret sauce" of the subculture.

The Bottom Line

This paper isn't about changing the whole world of design overnight. It's about showing that culture matters. You can't just teach a computer to be a colorist; you have to teach it to be a cultural colorist. By building a specific dataset and a smart two-step system, they proved that AI can learn to paint with the specific, rebellious, and vibrant colors of a specific group of people.

1. Problem Statement

Existing research and tools for algorithmic color palette generation and image colorization largely ignore the cultural context of color usage.

Cultural Variance: Colors carry different semantic meanings across cultures (e.g., red/green combinations are traditionally "unpleasant" in Chinese culture but represent "cool/rebellious" styles in Chinese Youth Subculture or CYS).
Limitations of Current Models: Standard models (like Adobe Color or generic GANs) rely on general color theory or broad datasets, failing to capture the specific aesthetic and semantic nuances of niche subcultures.
Gap: There is a lack of open datasets and generative frameworks specifically designed to learn and replicate the unique color logic of specific cultural groups, particularly the vibrant Chinese Youth Subculture (CYS) popular among Gen Z.

2. Methodology

The authors propose an open-source, culture-inspired multi-modal generative framework consisting of a unique dataset and a two-stage neural network architecture.

A. Dataset Construction (CYS Dataset)

Source: Manually collected 7,051 images (posters, album covers, illustrations) from websites targeting CYS.
Selection: Three designers curated 1,263 high-quality images with strong CYS stylistic elements.
Annotations:
- Color Palettes: 5-color palettes per image, manually selected by designers based on visual impact and CYS semantics (not just color proportion).
- Text: 2,535 unique Chinese adjectives, nouns, and verbs extracted from image descriptions.
- Categories: 14 subculture categories (e.g., punk, hiphop, techno).
Statistical Validation: A comparison with the generic PAT dataset (Bahng et al.) via HSL distribution analysis and T-tests revealed that CYS colors possess more diverse hues but lower saturation and lightness compared to generic datasets.

B. Framework Architecture

The system utilizes two separately trained Conditional Generative Adversarial Networks (cGANs) based on the Least-Squares GAN (LSGAN) formulation:

Color Palette Generation Network:
- Input: Multi-modal context including Grayscale Image ( $E_{image}$ ), Text Description ( $E_{text}$ ), and Category ( $E_{category}$ ).
- Mechanism: Uses a fusion function with learnable weights ( $\lambda$ ) to combine inputs into a context vector $c_1$ .
- Process: Generates colors sequentially. The discriminator determines if a generated color is the logical "next" color in the palette based on previous colors fused with the context.
- Output: A 5-color palette.
Image Colorization Network:
- Input: The generated 5-color palette and the same multi-modal context (with adjusted weights, prioritizing the image).
- Process: Applies the generated palette to the input grayscale image to produce a CYS-styled colored image.

C. Human-in-the-Loop System

A demo system was built to allow user interaction. Users can input text, upload images, and select categories. Crucially, the system records user adjustments to the generated palettes, creating a feedback loop to continuously improve the algorithm.

3. Key Contributions

CYS Dataset: Creation of the first open-source dataset specifically for Chinese Youth Subculture, containing 1,263 images with paired 5-color palettes, descriptive text, and categorical labels.
Multi-modal Generative Framework: Development of a novel cGAN architecture that integrates image, text, and category inputs to generate culture-specific color palettes and perform conditional colorization.
Interactive Demo System: A working prototype demonstrating the framework with a human-in-the-loop principle for iterative algorithm evolution.
Open Source Release: The dataset, code, and demo system are publicly available to facilitate further research in culture-aware design.

4. Results and Evaluation

The framework was evaluated through two primary perspectives:

Multi-modal Control (Qualitative/Quantitative):
- Experiments involved varying one input modality (text, image, or category) while keeping others constant.
- Result: The model demonstrated excellent controlled diversity. For instance, changing the text from "sun, fun" to "evil" resulted in significantly darker generated colors, proving the model successfully captures semantic-text-to-color relationships.
User Study (CYS Style Verification):
- Setup: 20 keywords were used to generate palettes for 20 neutral images. These were compared against palettes from Adobe Color (cultural neutral).
- Task: Four professional designers reviewed the pairs to identify which better reflected the CYS style.
- Outcome:
  - 75% of the generated palettes were preferred over the neutral ones.
  - 76% of the recolored images were judged as having a stronger CYS style.
- Conclusion: The framework successfully learned the specific color semantics of the CYS group.

5. Significance

Bridging Culture and AI: This work is the first to explicitly integrate cultural subculture data into automatic color generation, moving beyond generic aesthetic rules.
Design Innovation: It provides a tool for designers to rapidly prototype and explore styles rooted in specific cultural identities (specifically Gen Z trends), which is vital for businesses targeting this demographic.
Scalability: While focused on CYS, the framework is designed to be adaptable to any culture provided specific training data is available, offering a generalizable approach to "culture-aware" computer vision.
Practical Application: The system addresses the gap between traditional color theory and the evolving, rebellious, and vibrant visual language of modern youth subcultures.