Technical Summary: Beyond Translation: Cross-Cultural Meme Transcreation with Vision-Language Models

Problem Statement

Memes serve as a dominant form of online communication, yet their effectiveness is deeply rooted in cultural specificity. While literal translation preserves surface meaning, it often fails to capture the communicative intent, humor, and cultural resonance required for a meme to succeed in a new context. This challenge, defined as transcreation, requires the coordinated adaptation of both text and images to align with target-culture norms, references, and aesthetic preferences.

Current Vision-Language Models (VLMs) and generative models demonstrate strong multimodal capabilities, yet systematic frameworks, datasets, and evaluations for cross-cultural meme generation remain limited. It is unclear how well existing models can perform culturally grounded transcreation, how performance varies across cultural directions (e.g., US to Chinese vs. Chinese to US), and how such systems should be rigorously evaluated.

Methodology

Hybrid Transcreation Framework

The authors propose a hybrid framework that separates culture-invariant elements from those requiring adaptation. The approach is grounded in three principles:

1. Preserve universal elements: Retain core humor mechanisms (e.g., irony, exaggeration), high-level emotional intent, and common meme formats.
2. Adapt culture-specific elements: Replace culturally grounded references (pop culture, idioms, visual symbols) with culturally appropriate alternatives rather than literal translations.
3. Maintain intent consistency: Ensure the meme's communicative goal remains intact even as textual and visual inputs change.

The framework operates via a three-stage pipeline:

• Stage 1: Cultural Analysis and Caption Generation: Using LLaVA 1.6 (13B), the system analyzes the source meme to identify cultural references and humor mechanisms. It extracts the underlying intent, maps source concepts to target equivalents, and generates a culturally adapted caption in the target language. This stage also provides high-level visual recommendations (mood, background).
• Stage 2: Visual Template Generation: Using FLUX.1 Schnell (a diffusion-based image generation model), the system generates culturally adapted meme templates based on the Stage 1 recommendations. The goal is to produce meme-appropriate visuals (e.g., replacing Western celebrities with symbolic or animal-based representations common in the target culture) rather than photorealism.
• Stage 3: Final Assembly: Using the Pillow library, the system automatically combines the transcreated captions with the generated visual templates. This step handles font selection, text placement, and layout conventions specific to the target culture (e.g., denser layouts for Chinese text).

Dataset: MemeXGen

To support controlled study, the authors introduce MemeXGen, a large-scale bidirectional dataset containing 6,315 original memes and 6,315 transcreated memes.

• Sources: Chinese memes are sourced from Xiaohongshu and Weibo; US memes are sourced from Reddit communities (r/memes, r/dankmemes).
• Structure: The dataset includes 3,165 Chinese→US pairs and 3,150 US→Chinese pairs.
• Filtering: Sensitive content (e.g., political references) and low-quality images were manually filtered. The retention rate was higher for Chinese memes (97.6%) than US memes (55.0%), reflecting stricter content moderation on Chinese platforms.
• Annotation: A subset of 628 memes was annotated for emotion (category and intensity) by three expert annotators, achieving strong agreement (Fleiss' κ = 0.869 for category).

Evaluation Methodology

The study employs both human and automated evaluation across six dimensions rated on a 5-point Likert scale:

1. Caption Quality: Clarity, tone, and meme-appropriate language.
2. Image Quality: Visual clarity, composition, and recognizability.
3. Synergy: How well image and text work together.
4. Cultural Fit: Appropriateness and relatability for the target culture.
5. Intent Preservation: Retention of the original message and emotional effect.
6. Overall Score: The average of the above dimensions.

Human Evaluation: Conducted by three bilingual, bicultural evaluators familiar with both US and Chinese meme cultures.
Automated Evaluation: Six state-of-the-art VLMs (including Qwen-VL-Max, LLaVA-v1.6, InternVL3, Qwen3-VL, and Aya-vision) were used to assess the transcreated memes.

Key Results

Cross-Cultural Performance (RQ1)

• Overall Quality: The proposed pipeline produces effective transcreations, with a mean human score of 4.07/5.0.
• Dimension Analysis: Caption Quality (4.20) and Synergy (4.23) received the highest ratings. Cultural Fit showed the widest variation among evaluators (3.39–4.57), highlighting the subjective nature of cultural authenticity.
• Automated vs. Human: Qwen-VL-Max emerged as the best automated evaluator, showing strong alignment with human judgments (mean Pearson r = 0.926). Other open-source models exhibited significantly weaker correlations (r ≤ 0.33), suggesting they struggle to evaluate creative, culturally grounded outputs.

Directional Asymmetries (RQ2)

A significant directional asymmetry was observed:

• US→Chinese transcreations significantly outperformed Chinese→US (4.48 vs. 3.93 out of 5.0).
• Causes: US memes often rely on globally recognizable characters and themes that are easier to localize. Conversely, Chinese memes frequently depend on context-specific wordplay and implicit cultural knowledge that lacks direct Western equivalents. Additionally, current VLMs are more heavily exposed to US-centric data during training.

Evaluation Framework (RQ3)

• Correlation: Qwen-VL-Max showed consistently strong alignment with all three human evaluators, particularly on Intent Preservation (r = 0.993).
• Limitations: Most other models failed to correlate with human judgments, indicating that evaluating cross-cultural multimodal content requires deeper multilingual and multicultural grounding than currently provided by most open-source VLMs.

Qualitative Findings

• Success Patterns: High-quality transcreations utilized universally applicable character archetypes, retained emotional context while incorporating cultural specifics, and matched visual and textual unity.
• Failure Patterns: Common errors included formal speech that dampened the meme vibe, visual disconnects, failure to preserve humor mechanisms, and complete synergy breakdown between caption and image.

Significance and Contributions

The paper makes three primary contributions to the field of multimodal generation and cross-cultural AI:

1. Hybrid Framework: It proposes a practical framework for meme transcreation that balances intent preservation with cultural adaptation, offering guidance for culturally grounded adaptation.
2. Bidirectional Dataset: It introduces the first large-scale, bidirectional meme transcreation dataset (MemeXGen) spanning Chinese and US cultures, enabling direct comparison of transcreation quality across directions.
3. Empirical Study & Evaluation: It presents a systematic empirical study revealing that transcreation quality is direction-dependent and that specific aspects of internet humor (imagery, text style, emotion) transfer unevenly. Furthermore, it establishes a multi-dimensional evaluation framework comparing human and automated assessments, demonstrating that current automated metrics are insufficient for capturing cultural fit and humor without advanced, culturally grounded models.

The authors conclude that while current VLMs can perform cross-cultural meme transcreation to a limited extent, significant challenges remain in bridging cultural gaps, particularly in the Chinese→US direction, and in developing evaluation metrics that accurately reflect human cultural judgment.