InCoder-32B: Code Foundation Model for Industrial Scenarios

The paper introduces InCoder-32B, a 32-billion-parameter code foundation model trained from scratch with a specialized curriculum to achieve competitive performance on general programming tasks while establishing strong open-source baselines for complex industrial scenarios involving chip design, GPU optimization, embedded systems, compilers, and 3D modeling.

The Gist

Meet InCoder-32B: The Master Craftsman of the Digital Factory

Imagine you have a brilliant, all-knowing assistant who can write code for building websites, writing mobile apps, and playing video games. This assistant is great at general tasks. But now, imagine you ask this assistant to design the actual engine of a rocket, optimize the traffic flow inside a supercomputer chip, or program a tiny robot that lives inside a medical device.

Suddenly, the assistant starts making mistakes. Why? Because building a website is like painting a picture; building a rocket engine is like performing brain surgery. One requires creativity; the other requires absolute precision, knowledge of physics, and strict adherence to safety rules.

InCoder-32B is a new AI model designed specifically to be that "brain surgeon" of the coding world. It's not just a generalist; it's a specialist trained for the toughest, most critical jobs in the tech industry.

Here is a simple breakdown of how it works and why it matters.

---

1. The Problem: The "Generalist" Gap

Think of current popular coding AIs as Jack-of-all-trades, master of none. They have read millions of books on how to build a house (general software), but they've never actually laid a brick in a nuclear power plant (industrial hardware).

When you ask a general AI to write code for a computer chip (Verilog) or a graphics card (CUDA), it often hallucinates. It might write code that looks correct but would cause the chip to overheat, the robot to crash, or the simulation to fail. It doesn't "feel" the hardware constraints.

InCoder-32B was built to fix this. It is the first AI specifically trained to understand the "physics" of code: timing, memory limits, and hardware rules.

2. The Training: From Student to Apprentice to Master

The researchers didn't just feed InCoder-32B more books. They created a three-stage "boot camp" to turn it into an industrial expert.

Stage 1: The Library (Pre-Training)

• The Analogy: Imagine a student reading every textbook in a university library.
• What happened: The team gathered code from public repositories, but they didn't just grab random code. They used special filters to find the "hard stuff": code for chips, GPUs, and embedded systems. They cleaned it up, removed errors, and made sure it was high-quality. This gave the model a broad vocabulary of industrial terms.

Stage 2: The Simulation Lab (Mid-Training)

• The Analogy: Now the student goes to a flight simulator. They don't just read about flying; they practice in a virtual cockpit that mimics real turbulence and engine failures.
• What happened: This is the secret sauce. The team built virtual factories inside computers.
  • They created a virtual chip factory where code is tested against real hardware rules.
  • They built a virtual GPU lab where code is run on simulated graphics cards.
  • They created a virtual robot lab for embedded systems.
  • The Twist: They generated millions of "What if?" scenarios. What if the memory is full? What if the timing is off by a nanosecond? The AI learned to reason through these problems, not just guess. It also learned to handle huge amounts of data (up to 128,000 words of context), like reading a whole blueprint before drawing a single line.

Stage 3: The Internship (Post-Training)

• The Analogy: The student is now an intern in a real factory. They try to fix a broken machine. If they fail, the machine gives them an error message. The student reads the error, fixes the mistake, and tries again.
• What happened: The AI was given real-world tasks. When it wrote code that failed, the system didn't just say "Wrong." It showed the AI the error log, the simulation crash, or the timing violation. The AI learned to look at its own mistakes, understand why they happened, and fix them. This "feedback loop" taught it how to debug like a human engineer.

3. What Can It Do? (The Superpowers)

InCoder-32B is like a Swiss Army knife for the most complex engineering jobs. It handles four main "industries":

1. Chip Design (The Brain Builders): It can write code for the tiny circuits inside your phone or a supercomputer. It understands how to arrange logic gates so the chip doesn't melt.
2. GPU Optimization (The Speed Demons): It writes code that makes graphics cards run faster. Think of it as tuning a race car engine to squeeze out every last drop of speed.
3. Embedded Systems (The Tiny Robots): It programs the microchips inside washing machines, cars, and medical devices. These chips have very little memory, so the code must be incredibly efficient.
4. 3D Modeling (The Digital Architects): It writes scripts to build 3D objects for manufacturing, ensuring the shapes are mathematically perfect for printing or machining.

4. The Results: Beating the Giants

The researchers tested InCoder-32B against the biggest, most famous AI models (like those from Google, OpenAI, and Alibaba).

• On General Tasks: It performed just as well as the giants, even though it is smaller and cheaper to run.
• On Industrial Tasks: It crushed the competition. In areas like chip design and GPU optimization, it was the best open-source model available, often beating even the massive, expensive "closed" models that cost millions to use.

5. Why Does This Matter?

Imagine if you could hire a senior engineer who knows how to fix a jet engine, program a robot, and design a microchip, all for the price of a standard software assistant.

InCoder-32B makes this possible. It bridges the gap between "writing a script to scrape a website" and "designing the hardware that powers the internet." It proves that with the right training (simulations, feedback, and specialized data), AI can move from being a creative writer to a reliable industrial engineer.

In short: InCoder-32B is the AI that doesn't just write code; it understands the machine the code runs on. It's the difference between a painter who draws a picture of a car and a mechanic who can actually build one.

Technical Summary

Technical Summary: InCoder-32B: Code Foundation Model for Industrial Scenarios

1. Problem Statement

While Large Language Models (LLMs) have achieved remarkable success in general software engineering tasks (e.g., web development, algorithmic coding), they face significant performance degradation in industrial code scenarios. These scenarios include chip design (Verilog/RTL), GPU kernel optimization (CUDA/Triton), embedded systems, compiler optimization, and 3D modeling.

The core challenges preventing general code LLMs from succeeding in these domains are:

• Hardware Semantics: Industrial code requires reasoning about physical constraints (e.g., GPU grid dimensions, register layouts, timing constraints) rather than just logical correctness.
• Strict Constraints: Industrial tasks often involve hard limits (e.g., memory bandwidth, synthesis area, real-time deadlines) where a logically correct but physically invalid solution is a failure.
• Verification Complexity: Correctness in these domains cannot be determined by simple unit tests; it requires simulation, formal verification, or execution on real hardware.
• Data Scarcity: Public repositories are heavily biased toward web applications and high-level software, lacking the depth of specialized industrial code (e.g., RTL, firmware, kernel drivers) needed for training.

Existing models often fail on these tasks, with top models achieving only ~28% success rates on Triton operator generation and failing formal equivalence checks on Verilog code.

2. Methodology: The Code-Flow Pipeline

The authors introduce InCoder-32B, a 32-billion parameter code foundation model designed specifically to bridge the gap between general coding and industrial engineering. The model is trained using a three-stage Code-Flow pipeline:

Stage 1: Pre-training & Data Curation

• Data Collection: The team constructed a specialized dataset by curating industrial code from public repositories, technical literature (via OCR), and domain-specific web sources. They employed a three-step recall strategy (rule-based, classifier-based, and semantic retrieval) to maximize coverage of Verilog, CUDA, embedded C, and CAD scripts.
• Cleaning & Refinement: Data underwent rigorous filtering for licenses, PII, and validity. Crucially, they applied AST comparison and re-compilation to ensure syntactic and semantic correctness before training.
• Architecture: InCoder-32B uses a standard decoder-only Transformer architecture (32B parameters, 64 layers, 5120 hidden size) trained with autoregressive language modeling and Fill-in-the-Middle (FIM) objectives.

Stage 2: Mid-Training (Context Extension & Reasoning)

This stage transforms the base model into an "industrial-aware" foundation.

• Progressive Context Scaling: The context window is extended from 8K to 32K, and finally to 128K tokens. This allows the model to handle large hardware projects, cross-module dependencies, and extended debugging sessions.
• Synthetic Industrial Reasoning: To address the scarcity of high-quality reasoning data, the authors generated synthetic QA pairs and agent trajectories.
  • Simulation-Grounded: Data synthesis involves consulting with hardware engineers to define scenarios, generating seed code, and using automated verification (simulators, compilers) to validate the reasoning traces.
  • Agent Trajectories: The model is trained on "Thought-Action-Observation" cycles where it interacts with hardware simulators, synthesis tools, and formal verification engines to debug and repair code.

Stage 3: Post-Training (Execution-Grounded Verification)

• SFT Construction: The team constructed 2.5 million Supervised Fine-Tuning (SFT) samples directly from real industrial tasks.
• Closed-Loop Repair: Unlike standard SFT, this pipeline captures failed attempts, the execution feedback (compiler errors, simulation mismatches, timing violations), and the repaired solution. This teaches the model to diagnose errors based on real tool outputs.
• Variants: The process yields both an instruction-tuned variant and a "thinking" variant capable of analytical reasoning.

3. Key Contributions

1. First Unified Industrial Code LLM: InCoder-32B is the first model to unify capabilities across chip design, GPU optimization, embedded systems, compiler optimization, and 3D modeling within a single 32B parameter architecture.
2. Comprehensive Evaluation Benchmark: The authors assembled the most extensive industrial code evaluation suite to date, comprising 9 industrial benchmarks across 4 specialized domains (e.g., VeriScope, RealBench, KernelBench, CAD-Coder) alongside 14 general code benchmarks.
3. Simulation-Grounded Training: A novel pipeline that reconstructs industrial environments (EDA tools, GPU simulators, Renode for embedded) to generate training data where correctness is verified by the same tools used in production.
4. Ablation Insights: The study reveals that:
   • Repository transition data (commits) outperforms static snapshots for planning.
   • Mid-training reasoning trajectories significantly improve robustness under distribution shifts.
   • Thinking paths (Chain-of-Thought) unlock emergent capabilities absent in standard instruction tuning.

4. Experimental Results

InCoder-32B was evaluated against leading open-weight (e.g., Qwen3-Coder, DeepSeek-V3.2, Kimi-K2) and proprietary models (e.g., Claude-Sonnet-4.6).

General Code Performance

• Achieved highly competitive results on general benchmarks (EvalPlus, LiveCodeBench, SWE-bench Verified) despite its compact 32B size.
• SWE-bench Verified: 74.8% (competitive with much larger models).
• LiveCodeBench: 49.14%.
• BFCL (Tool Use): 60.99%.

Industrial Code Performance

InCoder-32B established strong open-source baselines, often outperforming models with significantly more parameters and even proprietary closed-source models in specific domains:

• Chip Design (Verilog):
  • RealBench (Module Level): 83.3% Syn@5 (Best open-weight result).
  • VeriScope: 80.7% Score.
  • VeriRepair: 80.0% Fix rate.
• GPU Optimization:
  • KernelBench: Outperformed all open-weight baselines across L1, L2, and L3 levels.
  • TritonBench: Achieved 18.5% G-call and 19.3% G-exe, surpassing larger models.
• 3D Modeling (CAD):
  • CAD-Coder: Achieved 82.0% compilation success and 53.5% IoU, surpassing the proprietary Claude-Sonnet-4.6 (77.0% Comp, 32.4% IoU).
• Embedded & Compiler:
  • EmbedCGen: 35.2% pass rate.
  • SuperCoder: 91.0% accuracy on assembly superoptimization.

5. Significance and Impact

• Bridging the Gap: InCoder-32B demonstrates that a single, relatively compact model can master the diverse, high-stakes requirements of industrial engineering, moving beyond the "generalist" limitations of current code LLMs.
• Practical Industrial Adoption: By training on execution-grounded data and simulating real-world toolchains, the model produces code that is not just syntactically correct but functionally and physically valid for deployment in hardware and systems.
• Open-Source Leadership: The release of InCoder-32B and its comprehensive benchmarks provides the community with a powerful, open-weight baseline for industrial AI, reducing reliance on proprietary black-box models for critical engineering tasks.
• Future Direction: The paper highlights that while significant progress has been made, challenges remain in handling rare APIs and complex hardware semantics, suggesting that future advancements will require even tighter integration between LLMs and domain-specific verification tools.

In summary, InCoder-32B represents a paradigm shift in code intelligence, moving from general software assistance to specialized industrial engineering capability through a rigorous, simulation-grounded training methodology.