CARMEN: CORDIC-Accelerated Resource-Efficient Multi-Precision Inference Engine for Deep Learning

Technical Summary of CARMEN: CORDIC-Accelerated Resource-Efficient Multi-Precision Inference Engine for Deep Learning

Problem Statement
Deep learning inference on edge devices faces significant challenges regarding area and power constraints. While Multiply-Accumulate (MAC) operations dominate computational workloads (approx. 90%), activation functions (AFs) contribute minimally (2–5%). Existing accelerators often allocate substantial hardware resources to dedicated AF units, leading to poor utilization (up to 84% idle cycles) and significant area overhead. Furthermore, the trade-off between computational accuracy and hardware efficiency is typically fixed at design time. Static approximation schemes or fixed CORDIC iteration depths either incur unnecessary latency for error-tolerant layers or compromise accuracy for sensitive computations. Prior approaches utilizing fixed-point quantization, CORDIC, or logarithmic approximations often introduce accuracy degradation (up to 4%) or suboptimal energy efficiency.

Methodology
The authors propose CARMEN, a runtime-adaptive, CORDIC-accelerated multi-precision vector engine designed to address workload imbalances and static accuracy-efficiency trade-offs. The architecture consists of five key components:

1. Runtime-Adaptive Iterative CORDIC MAC: Unlike conventional pipelined designs that replicate hardware, CARMEN utilizes a low-resource iterative CORDIC-based MAC unit. It reuses arithmetic resources (adders, multiplexers, shift units) across iterations. Crucially, the iteration depth—which directly governs accuracy—can be dynamically tuned at runtime without hardware modification. This allows seamless switching between approximate and accurate execution modes based on layer criticality.
2. Time-Multiplexed Multi-Activation Function (Multi-AF) Block: To mitigate the inefficiency of dedicated AF hardware, the design employs a time-multiplexed approach. A single CORDIC core operates in hyperbolic rotation mode to compute complex functions (Sigmoid, Tanh), while softmax is realized via exponentiation and normalization. Simpler functions (ReLU, Swish, etc.) are handled via bypass logic. This shared datapath supports seven distinct activation functions with minimal area overhead (<4%).
3. Supporting Peripherals: The engine includes an on-the-fly AAD pooling unit, a normalization unit, and an AXI-based off-chip memory interface. A centralized control engine manages configuration registers, status flags, and data movement to ensure high hardware utilization across different layers.
4. Scalable Vector Engine: The system is built on a hierarchical, synthesizable generator framework, allowing scalable instantiation of Processing Elements (PEs) for both edge and high-performance computing scenarios.

Key Contributions

1. Dynamic Accuracy-Efficiency Trade-off: A runtime-adaptive MAC unit that supports approximate and accurate modes, enabling up to a 33% reduction in computation cycles with less than 2% accuracy loss.
2. Resource-Efficient Architecture: A reconfigurable vector engine integrating MAC, time-multiplexed multi-AF (supporting ReLU, GELU, SoftMax, Tanh, Sigmoid, Swish, SELU), pooling, and normalization units with efficient resource sharing.
3. Comprehensive Evaluation: Extensive FPGA and ASIC evaluations demonstrating high compute density and energy efficiency, validated through system-level deployment.

Results

• FPGA Implementation (Pynq-Z2/VC707): The design achieves 6.43 GOPS/W energy efficiency at 0.53 W with zero DSP usage. For a 64-PE configuration, it requires only 24 LUTs and 22 FFs per MAC unit, roughly half the resources of comparable CORDIC-based designs.
• ASIC Implementation (28 nm CMOS):
  • 64-PE Configuration: Occupies 0.43 mm² (the smallest footprint among compared accelerators) with 3.84 TOPS/W energy efficiency.
  • 256-PE Configuration: Delivers 4.83 TOPS/mm² compute density and 11.67 TOPS/W energy efficiency. This represents a 1.75× improvement in compute density and 1.5× improvement in energy efficiency over the closest competitors.
  • Power Savings: The iterative MAC unit achieves a 21% reduction in power per MAC stage compared to prior art.
• System-Level Validation: Deployed on a Pynq-Z2 platform for real-time object detection (TinyYOLO-v3), the system achieves 154.6 ms latency at 0.43 W, outperforming prior academic accelerators (e.g., LPRE, ILM) and commercial edge platforms (NVIDIA Jetson Nano, Raspberry Pi).
• Accuracy: At 8-bit fixed-point precision, CARMEN maintains accuracy within 2% of the conventional FP32 baseline across various DNN models (LeNet-5, ResNet-18, VGG-16, CaffeNet).

Significance
The paper claims that CARMEN effectively bridges the gap between computational accuracy and hardware efficiency by leveraging the inherent properties of CORDIC iteration depth. By dynamically adjusting precision at runtime and time-multiplexing activation function hardware, the architecture significantly reduces area and power overhead while maintaining high throughput. The results validate that a resource-constrained, iterative approach can outperform static, fully parallel, or DSP-heavy designs in terms of energy efficiency and area density, making it a viable solution for edge AI deployment. Future work aims to extend this into a compiler-assisted framework for automated layer-wise precision tuning and transformer-based workloads.