Technical Summary: AuroraEdge-V-2B

Problem Statement

While Visual Large Language Models (VLLMs) offer superior generalization and flexibility compared to traditional Deep Learning Models (DLMs) in industrial settings, their deployment on edge devices faces significant hurdles. Existing VLLMs typically suffer from:

1. High Computational Cost: Large parameter counts and the generation of excessive visual tokens during decoding create substantial floating-point operation (FLOP) burdens.
2. Latency: The sheer volume of visual tokens (e.g., 576 tokens per image in LLaVA-1.5) hinders real-time response, making them slower than custom DLMs.
3. Resource Constraints: Industrial edge environments often lack the substantial computational resources required to run state-of-the-art (SOTA) VLLMs.

The core challenge is to develop a VLLM that retains strong performance across diverse tasks while being compact enough for edge deployment and fast enough for real-time industrial applications.

Methodology

The authors propose AuroraEdge-V-2B, a 2-billion parameter VLLM designed for edge deployment. The model is built upon the LLaVA architecture but introduces specific modifications to balance efficiency and performance:

1. Architecture Modifications

• Base Components: The model utilizes a pre-trained SigLIP2 visual encoder (supporting dynamic resolutions via Patch n'Pack and FlexiViT) and a Qwen2.5-1.5B LLM backbone.
• Token Compressor: To reduce the computational burden of visual tokens, a Token Compressor module is inserted after the projector. It reduces the visual token sequence from 256 tokens (extracted from the image) down to a fixed 64 tokens. The authors experimentally determined that a Multi-Layer Perceptron (MLP) is the most effective compression method compared to Conv2d or MaxPool2d.
• Fusion Module: To compensate for the information loss inherent in token compression, a Fusion Module injects the original (pre-compression) visual information into the text tokens. The authors compared three fusion strategies:
  • Cross-Attention: Standard cross-attention between text and visual tokens.
  • Decoder: A single-layer transformer decoder.
  • Combined: A summation of the Cross-Attention and Decoder outputs.
    The Combined approach yielded the best performance and was selected for the final model.

2. Training Strategy

The model undergoes a rigorous three-stage training process:

• Stage 1 (Vision Training): The LLM and Visual Encoder are frozen. Only the projector, token compressor, and fusion module are trained using image-caption data to align modalities.
• Stage 2 (LLM Finetuning): The Visual Encoder is frozen. The LLM backbone, projector, compressor, and fusion module are updated using Visual Question Answering (VQA) data to enhance visual comprehension.
• Stage 3 (Vision-Text Joint Training): All parameters are updated using a mix of detailed image-caption data and complex VQA data.

The training dataset combines open-source resources (e.g., LLaVA-Pretrain, COCO, GQA, OKVQA) with a synthesized pipeline that uses GPT models to generate diverse queries and curation steps to ensure quality.

Key Contributions

1. AuroraEdge-V-2B Model: Introduction of a compact, 2B-parameter VLLM specifically optimized for edge deployment, offering a balance between speed and accuracy.
2. Compression-Fusion Method: A novel technique that reduces visual tokens by 75% (from 256 to 64) to halve inference FLOPs, while simultaneously using a fusion module to inject visual context back into text tokens, preserving performance.
3. Rapid Construction Framework: A reproducible, data-efficient methodology for building small VLLMs using pre-trained LLMs and VEs, demonstrating that high performance can be achieved without massive parameter counts.

Empirical Results

The model was evaluated on 11 widely recognized benchmarks (including ScienceQA, VQAV2, MMBench, and MME) and compared against SOTA models with similar parameter scales (Qwen2-VL-2B, Qwen2.5-VL-3B, InternVL-2.5-2B).

• Performance: AuroraEdge-V-2B achieved state-of-the-art (SOTA) scores on 9 out of 11 benchmarks, outperforming models with equal or larger parameter counts.
• Efficiency:
  • FLOPs: The model requires significantly fewer floating-point operations (less than 16% of the most computationally efficient baseline in the comparison).
  • Latency: On a single NVIDIA GeForce RTX 3090 GPU, the model demonstrated an inference speed 3 times faster than its competitors (40ms latency vs. ~116–143ms for others).
• Ablation Studies: Experiments confirmed that the "Combined" fusion method outperforms standalone Cross-Attention or Decoder methods, and that the fusion stage is critical, as models without fusion (Compress-only) showed significantly degraded performance.

Significance and Claims

The paper claims that AuroraEdge-V-2B addresses the critical gap between the theoretical capabilities of VLLMs and their practical usability in industrial edge environments. By demonstrating that a 2B-parameter model can outperform larger counterparts in both speed and accuracy, the work suggests a viable path toward replacing traditional, domain-specific DLMs with more flexible VLLMs in production.

The authors position this work as a step toward making VLLMs "faster and stronger" for real-world applications, highlighting that the proposed compression-fusion method allows for real-time responses without sacrificing the robustness required for industrial tasks. They acknowledge current limitations, such as the lack of video support and fixed hyperparameters, but assert that this architecture provides a strong foundation for future optimization in industrial AI deployment.