JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence

1. Problem Statement

The field of neural code intelligence has traditionally focused on text-based source code. However, modern applications increasingly require the generation and manipulation of rich visual outputs (e.g., data visualizations, interactive web UIs, scientific animations). Current approaches face two primary bottlenecks:

• Modeling Limitations: Existing models often treat tasks in isolation (e.g., a specific model for "chart-to-code" and another for "WebUI generation"), failing to generalize across domains or harmonize programmatic logic with visual expression.
• Data Scarcity: There is a critical lack of high-quality, large-scale multimodal code data (Instruction, Code, Visual Output triplets). Synthesizing such data is difficult due to the need for rigorous validation (execution engines) and the heterogeneity of visual outputs (static charts vs. complex animations).

2. Methodology

The authors propose a unified framework comprising a data synthesis toolkit and foundational models (JANUSCODER and JANUSCODERV).

A. Data Synthesis Toolkit & JANUSCODE-800K

To overcome data scarcity, the authors developed a complete pipeline to curate JANUSCODE-800K, the largest multimodal code corpus to date (800k samples). The pipeline involves:

1. Data Sourcing: Aggregating raw data from public repositories (StackV2), web corpora, Wolfram Demonstrations, and competitive programming.
2. Multi-Strategy Synthesis:
   • Guided Evolution: Iteratively evolving existing instruction-code pairs using high-level concepts (e.g., "add a widget") to increase complexity.
   • Re-contextualization: Analyzing code to generate more precise, descriptive natural language instructions.
   • Reverse Instruction: Transforming raw code snippets into instruction-following pairs.
   • Bidirectional Translation: Translating logic between semantically related domains (e.g., R/Mathematica to Manim) to transfer knowledge.
3. Quality Control & Reward Modeling:
   • Execution Validation: All generated code is run in sandboxed environments (Python, Playwright, Manim, Wolfram Engine) to ensure executability.
   • VLM/LLM Reward Modeling: A reward model (using Qwen2.5-VL-72B and Qwen3-235B) evaluates the alignment between the instruction, code, and the resulting visual output across four dimensions: Task Relevance, Task Completion, Code Quality, and Visual Clarity. Only high-scoring samples are retained.

B. Model Architecture

The paper introduces two unified models:

• JANUSCODER: A text-centric model trained on instruction-code pairs.
• JANUSCODERV: A vision-centric unified model trained on the full multimodal corpus (Instruction, Code, Image). It utilizes backbones like Qwen3-8B/14B and InternVL3.5-8B.
• Unified Interface: Unlike specialized models, JANUSCODER supports diverse tasks in a single interface:
  • Text-to-Code (e.g., "Generate a plot of X").
  • Image-to-Code (e.g., "Recreate this chart").
  • Code Editing (e.g., "Modify this UI based on the screenshot").
  • Dynamic Visualization (e.g., "Create an animation explaining Attention").

3. Key Contributions

1. Versatile Data Synthesis Toolkit: A novel pipeline that leverages cross-modal synergies (e.g., using scientific code to generate animation data) and automated reward modeling to produce high-fidelity multimodal datasets.
2. JANUSCODE-800K: The release of the largest and most diverse multimodal code intelligence corpus, covering Python visualizations, WebUIs, SVGs, scientific demonstrations, and Manim animations.
3. Unified Foundational Models (JANUSCODER/V): The first models to establish a visual-programmatic interface, capable of handling text-centric and vision-centric tasks within a single architecture, bridging the gap between symbolic logic and visual perception.
4. DTVBENCH Benchmark: A new benchmark for evaluating Dynamic Theorem Visualizations using Manim and Wolfram Mathematica, focusing on executability, code similarity, and instruction alignment.

4. Experimental Results

The models were evaluated on seven benchmarks, including established ones (PandasPlotBench, ChartMimic, WebCode2M) and new ones (DTVBENCH, InteractScience).

• Performance vs. Commercial Models: The JANUSCODER series (7B–14B parameters) approaches or exceeds the performance of leading commercial models like GPT-4o.
  • Example: On ChartMimic, JANUSCODERV-8B achieved a score of 74.20 (Customized Mimic), significantly outperforming GPT-4o (67.42).
  • Example: On WebCode2M, JANUSCODER-7B achieved a TreeBLEU score of 0.25 (Short), surpassing GPT-4o (0.15), indicating superior structural correctness in generated HTML.
• Generalization: The unified models outperform specialized models (e.g., VisCoder) in their specific domains while maintaining strong general coding capabilities (tested on BigCodeBench and LiveCodeBench).
• Ablation Studies:
  • Removing cross-domain data (e.g., algorithm data) significantly hurt performance on specialized visual tasks, validating the synergy strategy.
  • Removing Reward Modeling led to a clear performance drop, proving that executability alone is insufficient for high-quality visual code.

5. Significance

• Paradigm Shift: The paper moves beyond "text-to-code" or "image-to-code" silos, proposing a unified visual-programmatic interface where models understand the deep relationship between code logic and its visual manifestation.
• Scalability: The demonstrated ability of 7B–14B open-source models to rival 100B+ parameter commercial models suggests that data quality and synthesis strategies are more critical than sheer parameter count for multimodal code intelligence.
• Community Impact: By releasing JANUSCODE-800K and the JANUSCODER models, the authors provide a strong foundation for future research in automated UI generation, scientific visualization, and interactive content creation, reducing the engineering burden of data curation for the community.

In conclusion, JANUSCODER represents a significant leap forward in making AI capable of not just writing code, but understanding and generating the visual worlds that code creates.