💻 computer science

OneComp: One-Line Revolution for Generative AI Model Compression

OneComp is an open-source framework that automates the complex post-training compression of generative AI models into a reproducible, hardware-aware pipeline, enabling progressive mixed-precision quantization to bridge the gap between algorithmic innovation and efficient production deployment.

Original authors: Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida, Yudai Fujimoto, Hiroki Tokura, Yamato Arai, Yoshiyuki Ishii, Yusei Kawakami, Genki Shikada, Achille Jacquemond, Yoshihiko Fujisawa, Katsuki Fujisawa, Taku

Published 2026-04-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuma Ichikawa, Keiji Kimura, Akihiro Yoshida, Yudai Fujimoto, Hiroki Tokura, Yamato Arai, Yoshiyuki Ishii, Yusei Kawakami, Genki Shikada, Achille Jacquemond, Yoshihiko Fujisawa, Katsuki Fujisawa, Takumi Honda, Akira Sakai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, incredibly intelligent library (a Generative AI Model) filled with billions of books. This library is so huge that it doesn't fit in a single room; in fact, it requires a warehouse the size of a city to store all the books. If you want to take this library on a road trip (deploy it on a phone or a small server), you can't fit it in your car.

The Problem:
Currently, to shrink this library, you have to hire a team of specialized librarians (experts). Each librarian has a different tool: one uses a sledgehammer to smash books into tiny pieces, another uses a vacuum to suck out the ink, and a third uses a photocopier to shrink the pages. The problem is that these tools don't talk to each other. You have to manually decide which tool to use for which book, how much to shrink it, and in what order. If you make a mistake, the library becomes unreadable gibberish.

The Solution: OneComp
The paper introduces OneComp, which is like an automated, self-driving moving truck for your AI library. Instead of hiring a team of experts to manually shrink the books, you just tell the truck, "Here is the library, and here is the size of my car." OneComp takes over and figures out the entire moving plan automatically.

Here is how it works, broken down into simple steps:

1. The "Smart Inspector" (AutoBit)

Before moving anything, OneComp sends out a smart inspector. This inspector doesn't treat every book the same.

The Analogy: Imagine packing for a trip. You know your heavy encyclopedia needs a sturdy, big box, but your light paperback novel can go in a small envelope.
What it does: OneComp looks at every part of the AI model. It realizes some parts are "fragile" (if you shrink them too much, the AI gets stupid) and some are "tough" (they can be shrunk a lot without losing meaning). It automatically decides: "This part gets 4 bits of space, that part gets 2 bits." This is called Mixed-Precision, and it saves space without breaking the brain.

2. The "Smooth Operator" (Preprocessing)

Sometimes, the books have weird, giant sticky notes or massive ink blots that make them hard to shrink.

The Analogy: Imagine trying to fold a shirt with a huge, stiff stain. It's hard to fold neatly. But if you first wash the shirt and smooth out the fabric, it folds perfectly.
What it does: OneComp performs a "pre-wash" on the model. It rearranges the data to remove these "stains" (outliers) so that when it shrinks the model later, the pieces fit together perfectly without losing information.

3. The "Three-Stage Moving Process" (The Pipeline)

This is the coolest part. OneComp doesn't just do one thing; it moves in three progressive stages, like upgrading your packing method as you get more time and money.

Stage 1: The "One-Box-at-a-Time" Move (Layer-wise PTQ)
- The Analogy: You have a small car. You pack one shelf of books at a time, tape it up, and move it to the car. You do this shelf by shelf. It's fast and fits in a tiny car, but because you packed them separately, the books might get a little jumbled.
- The Magic: OneComp has a special trick called QEP. It remembers that the previous shelf was packed roughly, so it adjusts the current shelf to compensate for the jumble. It fixes the mistakes as it goes.
Stage 2: The "Whole Room" Move (Block-wise PTQ)
- The Analogy: Now you have a slightly bigger van. Instead of packing shelf by shelf, you pack an entire room of books at once. You can see how the books in the "History" section interact with the "Science" section and arrange them so they fit better together.
- The Magic: This fixes the jumbling from Stage 1 by looking at groups of books together, ensuring they still make sense as a team.
Stage 3: The "Entire Warehouse" Move (Global PTQ)
- The Analogy: You have a massive truck. You load the entire library at once. You can rearrange the whole building to ensure every single book is in the perfect spot relative to every other book.
- The Magic: This is the most expensive and slowest method, but it produces the highest quality result. OneComp lets you stop at any stage depending on how big your truck is.

4. The "Final Polish" (Fine-Tuning)

Sometimes, after moving, a few books might be slightly bent.

The Analogy: You hire a professional bookbinder to do a quick, targeted repair on just the bent pages, without re-writing the whole book.
What it does: OneComp can add a tiny, lightweight "patch" (using a technique called QLoRA) to fix any remaining errors, making the model perform almost as well as the original giant version.

Why is this a "Revolution"?

Before OneComp, if you wanted to shrink an AI, you had to be a PhD-level engineer who knew exactly which tool to use for every single part of the model. It was like trying to build a house with a hammer, a saw, and a glue gun, but you had to invent the glue yourself.

OneComp changes the game by turning it into a "One-Line" command.

For the average user: You type one command, and it handles the complexity. It's like ordering a pizza; you don't need to know how to make the dough or grow the tomatoes.
For the expert: If you want to tweak the recipe, you can, but you don't have to.

The Bottom Line:
OneComp is the universal translator between the complex world of AI research and the real world of running AI on your phone or laptop. It takes the "brute force" out of AI compression and replaces it with smart, automated, step-by-step refinement, making powerful AI accessible to everyone, not just those with supercomputers.

1. Problem Statement

The deployment of foundation models (e.g., LLaMA, Qwen) is increasingly constrained by memory footprint, inference latency, and hardware costs. While Post-Training Quantization (PTQ) offers a solution by reducing parameter precision without retraining, the current landscape is fragmented. Practitioners face a complex ecosystem of:

Disjoint Algorithms: Various methods (Hessian-aware rounding, activation-aware scaling, rotation-based outlier suppression) address specific sub-problems but lack a unified workflow.
Manual Configuration: Selecting optimal bit-widths, calibration strategies, and hardware-specific execution regimes requires significant expert intervention.
Resource Constraints: There is a trade-off between optimization strength (global vs. local) and GPU memory availability. Existing tools often force a choice between low-quality, low-memory methods (layer-wise) or high-quality, high-memory methods (global), without a smooth transition.

OneComp aims to bridge the gap between algorithmic innovation and production-grade deployment by transforming this fragmented expert workflow into a reproducible, resource-adaptive, automated pipeline.

2. Methodology: The OneComp Pipeline

OneComp is an open-source framework that takes a model identifier and hardware constraints as input, automatically generating a quantization workflow. The pipeline is designed around three core principles: End-to-End Automation, Pivot-Based Refinement, and Extensibility.

A. Automated Investigation & Planning

AutoBit (Mixed-Precision Planner): Instead of uniform bit-width assignment, OneComp solves a constrained integer linear programming problem to allocate heterogeneous bit-widths to different modules (e.g., attention vs. MLP layers) based on a global memory budget. It uses activation-aware error estimation (via diagonal Hessian approximations) to prioritize sensitive layers.
Preprocessing: To mitigate outlier issues common in low-bit quantization, OneComp applies equivalent transformations:
- Channel-wise Scaling: (e.g., SmoothQuant) to shift difficulty from activations to weights.
- Rotation-based Transforms: (e.g., SpinQuant) using learned orthogonal matrices to distribute outlier energy, reducing the dynamic range required for quantization.

B. Resource-Adaptive Quantization Stages

The pipeline executes a progressive refinement strategy, where each stage improves upon the previous one. The user can stop at any stage based on available compute.

Layer-wise PTQ (The Pivot):
- Mechanism: Quantizes one linear layer at a time. It is the most memory-efficient regime, feasible on a single consumer GPU.
- Innovations:
  - QEP (Quantization Error Propagation): Corrects the quantization target of the current layer to compensate for errors accumulated from previously quantized upstream layers using a closed-form correction matrix.
  - LPCD (Submodule-Aware Coordinate Descent): Extends optimization to coupled submodules (e.g., Q-K projections, V-O projections, MLP gates) rather than isolated layers, resolving cross-layer inconsistencies.
- Formats:
  - JointQ: For 3–4 bits, jointly optimizes integer weights and scales (outperforming standard GPTQ).
  - DBF/MDBF: For 1–2 bits, uses structured binary factorization. MDBF improves upon DBF by replacing the rank-one magnitude envelope with a rank- $l$ envelope, allowing better modeling of magnitude distributions.
Block-wise PTQ (Refinement):
- Mechanism: Loads one Transformer block at a time.
- Techniques:
  - Block-wise Compensation: Optimizes remaining FP16 parameters within a partially quantized block to absorb errors from quantized sublayers.
  - Block-wise Distillation: Jointly refines quantized parameters of a full block using MSE distillation from a full-precision teacher.
  - Cross-block Optimization: Uses a sliding window to refine adjacent block pairs, capturing inter-block dependencies.
- Optimization: Uses a Smooth Straight-Through Estimator (STE) with progressive temperature annealing to enable gradient-based optimization of discrete quantization parameters.
Global PTQ (End-to-End):
- Mechanism: Loads the entire model for joint optimization.
- Loss Function: Combines Token-Level KL Divergence (aligning output distributions), Intermediate Hidden-State Alignment (preserving internal representations), and Entropy Regularization.
- Techniques: Employs Sharpness-Aware Minimization (SAM) and Progressive Layer Unfreezing to ensure stable convergence.
Optional Fine-Tuning:
- Supports QLoRA to freeze the quantized backbone and train low-rank adapters, enabling domain adaptation or recovery of residual quantization errors.

3. Key Contributions

Unified Framework: OneComp is the first open-source package to integrate investigation, mixed-precision planning, and multi-stage refinement (layer $\to$ block $\to$ global) into a single, automated pipeline.
Pivot-Based Architecture: By treating the initial layer-wise quantized model as a "pivot," OneComp ensures that every subsequent stage (block-wise, global) improves the same model, guaranteeing monotonic quality gains as resources increase.
Algorithmic Innovations:
- AutoBit: Automated, activation-aware mixed-precision allocation.
- QEP & LPCD: Novel error-propagation correction and submodule-aware optimization that significantly boost layer-wise PTQ performance without requiring global memory.
- JointQ & MDBF: Advanced formats for 3–4 bit and extreme 1–2 bit regimes, respectively.
Hardware Awareness: The system dynamically adapts the pipeline depth (stopping at layer-wise, block-wise, or global) based on available GPU memory.

4. Experimental Results

Evaluated on LLaMA-3 and Qwen families across various benchmarks (WikiText-2, ARC, PIQA, WinoGrande):

Mixed-Precision (AutoBit): Outperforms uniform quantization significantly. On LLaMA-3-8B at 4.00 bpw, uniform quantization failed (PPL 383), while AutoBit maintained PPL 7.04, nearly matching full precision.
Layer-wise Improvements: QEP and LPCD drastically reduced the performance gap between layer-wise and global PTQ. On LLaMA-3-8B (4-bit), QEP reduced PPL from 12.66 (Base GPTQ) to 6.66.
JointQ vs. GPTQ: JointQ consistently outperformed standard GPTQ in both perplexity and zero-shot accuracy, particularly in per-channel settings.
Extreme Low-Bit (1–2 bits): MDBF outperformed DBF across all models. At 1.00 BPW, MDBF preserved meaningful accuracy (e.g., 0.518 ACC on LLaMA-3-8B vs. 0.505 for DBF), proving structured binary factors are viable for edge deployment.
Monotonic Gains: Experiments confirmed that adding Block-wise and Global stages consistently improved accuracy across all bit-widths, validating the progressive refinement hypothesis.

5. Significance

OneComp represents a paradigm shift from manual, algorithm-specific quantization to automated, resource-adaptive model compression.

Democratization: It allows non-experts to deploy high-performance quantized models with a single command, while offering experts a flexible platform for research.
Economic Impact: By enabling efficient deployment on commodity hardware (single GPUs) and edge devices, it lowers the barrier to entry for foundation model usage.
Future Roadmap: The authors envision OneComp evolving into a comprehensive compression platform covering activation quantization, KV-cache compression, and structured pruning, aiming to reshape the economics of AI by prioritizing efficiency over brute-force scaling.

Availability: The framework is open-source and available at https://github.com/FujitsuResearch/OneCompression.