Bhasha-Rupantarika: Algorithm-Hardware Co-design approach for Multilingual Neural Machine Translation

Technical Summary: Bhasha-Rupantarika – Algorithm-Hardware Co-design for Multilingual Neural Machine Translation

Problem Statement
The paper addresses the challenge of deploying Multilingual Neural Machine Translation (MNMT) systems in resource-constrained environments, specifically targeting rural areas in India and IoT devices. Traditional MNMT approaches often rely on separate models for different language pairs or high-precision (FP32) computations, which demand significant computational power and memory. This creates a barrier for real-time translation on edge devices with limited resources. Furthermore, existing hardware accelerators often overlook the specific needs of NLP engines, focusing instead on vision or general deep learning tasks, leading to inefficiencies in area, power, and latency when handling transformer-based translation models.

Methodology
The authors propose "Bhasha-Rupantarika," a system designed through a tight algorithm-hardware co-design approach to enable efficient multilingual translation on edge platforms.

• Algorithmic Strategy:

  • Unified Model: Instead of chaining separate models (e.g., Indian Language $\to$ English $\to$ Foreign Language), the system utilizes a single unified bidirectional model based on NLLB-200 (No Language Left Behind). This model supports 200 languages, including low-resource Indian languages.
  • Quantization: The core algorithmic innovation involves aggressive sub-octet precision quantization. The study investigates deployment at FP8, INT8, INT4, and FP4 levels. A distilled 600M-parameter INT4 version of NLLB-200 is employed, utilizing a Transformer encoder-decoder architecture with Mixture-of-Experts (MoE) layers to manage parameter efficiency without increasing per-token compute costs.
  • Pipeline: The system integrates OpenAI Whisper Large-v3 for foreign language speech-to-text, AI4Bharat IndicConformer for Indian language speech-to-text, and Coqui XTTS v3/IndicParler TTS for text-to-speech synthesis, all orchestrated around the quantized NLLB translation core.

• Hardware Architecture:

  • Neural Processing Engine (NPE): A custom hardware accelerator was designed using SystemVerilog, featuring a Control Unit (CU), Memory Read/Write Units, a SIMD Matrix Multiply Engine (MME), and a Non-linear MIMD Vector Array (NMV).
  • Custom Compute Units:
    • MME: Implements a parameterized systolic array based on SIMD MAC (Multiply-Accumulate) operations. It supports multiple precisions (1$\times$BF16, 3$\times$FP8, 6$\times$FP4, 6$\times$INT4) in a single clock cycle using a novel RMMEC (Reconfigurable Multi-Mode Exponent and Mantissa Computation) block.
    • NMV: Handles non-linear activation functions (NAFs) such as Sigmoid, Tanh, ReLU, and SoftMax using a CORDIC-based FASST unit. This unit is designed to be reconfigurable and supports FP8 and BF16 precisions, addressing the high area cost of NAF hardware in commercial TPUs.
  • Dataflow: The architecture utilizes an output-stationary dataflow to maximize data reuse and minimize off-chip memory access latency.

Key Contributions

1. Unified Quantized Model: Demonstration that a single, distilled INT4 NLLB-200 model can effectively handle bidirectional translation between Indian and international languages, replacing complex multi-model pipelines.
2. Custom Hardware Accelerator: Development of a specialized NPE architecture with novel RMMEC and FASST units that support ultra-low precision (down to 4-bit) operations, specifically optimized for transformer workloads.
3. Co-design Validation: A comprehensive evaluation showing that algorithmic quantization (FP4/INT4) combined with custom hardware design yields significant gains in resource utilization and throughput compared to existing FPGA and ASIC solutions.

Results
The system was evaluated using a ZCU104 FPGA and simulated via Cadence Innovus (28nm CMOS) for ASIC metrics.

• Model Efficiency: The FP4 quantized model achieved a 4.1$\times$ reduction in model size (down to 0.56 GB) compared to FP32.
• Inference Performance: The system achieved a throughput of 66 tokens/second, representing a 4.2$\times$ speedup in inference speed and a 4.8$\times$ improvement in throughput compared to baseline expectations.
• Hardware Resource Utilization (FPGA):
  • LUTs: Reduced by 1.96$\times$ compared to the best State-of-the-Art (SoTA) NLP accelerator (OPU).
  • Flip-Flops (FFs): Reduced by 1.65$\times$.
  • Throughput: Achieved a 2.2$\times$ enhancement over OPU and 4.6$\times$ over HPTA.
• ASIC Metrics: The custom MAC and NAF units demonstrated significant reductions in area and power consumption compared to SoTA designs, with the MAC unit showing a 90% reduction in LUT usage compared to specific prior works.
• Accuracy: The system maintained satisfactory translation quality across bidirectional pairs (Indian $\leftrightarrow$ International languages) despite the aggressive quantization, as verified by qualitative outputs (e.g., "Model Verification and Validation").

Significance and Claims
The paper claims that Bhasha-Rupantarika provides a viable, deployable solution for multilingual AI in low-resource settings. By proving that ultra-low precision quantization (FP4/INT4) does not compromise translation capability when paired with specialized hardware, the work enables real-time translation on IoT devices and edge platforms. The authors emphasize that this approach is particularly significant for populations with limited formal education who need to engage with foreign languages for tourism or business. The study concludes that algorithm-hardware co-design is essential for balancing performance, accuracy, and resource constraints in edge NLP, offering a path toward energy-efficient, sustainable AI deployment in developing regions. The authors note that while the current work focuses on edge deployment, the scalability of this approach could potentially benefit high-performance cloud implementations in the future.