Retrospective: A CORDIC Based Configurable Activation Function for NN Applications

1. Problem Statement

The rapid evolution of AI workloads—from Deep Neural Networks (DNNs) and RNNs/LSTMs to modern Transformers and Vision Transformers (ViTs)—has created a significant bottleneck in hardware acceleration. While Multiply-Accumulate (MAC) units are highly optimized, non-linear Activation Functions (AFs) remain a critical constraint.

• Resource Inefficiency: Existing hardware often dedicates disproportionate area to AFs (e.g., Google TPUv4 allocates 20% of area for <2% of non-linear functions) or relies on Look-Up Tables (LUTs) which lack precision scalability.
• Lack of Flexibility: Traditional ASIC designs are often fixed to specific functions (e.g., ReLU or Sigmoid), making them inefficient for diverse workloads requiring functions like Swish, GeLU, or SoftMax.
• Precision vs. Area Trade-off: Existing approximation methods (Piecewise Linear, Taylor Series, Stochastic Computation) often suffer from high latency, quantization errors, or excessive control overhead, failing to balance reconfigurability with resource efficiency.

2. Methodology

The paper proposes a retrospective analysis of previous CORDIC (Coordinate Rotation Digital Computer) approaches and introduces a new, unified architecture called DA-VINCI (Dynamically Configurable Activation Function Core) and its integration into the NEURIC (Neuron Engine based on Reconfigurable Iterative CORDIC) compute unit.

Core Architecture: DA-VINCI

• Unified CORDIC Algorithm: The design utilizes a single hardware datapath capable of performing Circular, Linear, and Hyperbolic operations in both rotational and vector modes.
• Shift-and-Add Technique: Instead of complex multipliers or large LUTs, the core relies on simple Add/Sub, multiplexers (MUX), and logarithmic barrel shifters. This minimizes area and power.
• Dynamic Reconfigurability: The core supports runtime switching between multiple activation functions via control signals (sel_af):
  • Hyperbolic Mode (HR): Used for Sigmoid, Tanh, Swish, SoftMax, SELU, and GeLU.
  • Linear Mode (LV): Used for division operations required in Swish, SoftMax, and GeLU.
  • Additional Logic: Minimal overhead buffers and FIFOs are added to support ReLU and SoftMax accumulation without significant area penalty.
• Precision Scalability: The architecture supports 8-bit and 16-bit fixed-point (FxP) precision, with validation extending to 32-bit for error analysis.

System Integration: NEURIC Vector Engine

• The DA-VINCI core is integrated into a Layer-Multiplexed Vector Engine.
• This engine combines reconfigurable MAC units with the DA-VINCI AF unit, allowing a single processing element (PE) to handle both linear (MAC) and non-linear (AF) computations.
• The design supports both iterative (area-efficient) and pipelined (high-throughput) execution strategies.

3. Key Contributions

1. DA-VINCI Programmable AF Core: A novel core that supports Swish, SoftMax, SELU, GeLU, Sigmoid, Tanh, and ReLU with negligible overhead compared to single-function implementations.
2. NEURIC Compute Unit: A fundamental building block for AI accelerators featuring a reconfigurable 8/16-bit MAC + AF unit, enabling lightweight, layer-multiplexed AI cores.
3. Optimized CORDIC Implementation: The integration of HOAA (High-Order Area-Aware) techniques and optimized data paths reduces area by ~21% and power by ~33% compared to standard CORDIC designs.
4. Comprehensive Validation: Extensive empirical evaluation across FPGA (Zybo, Virtex-7) and ASIC (28nm/45nm) platforms, benchmarked against state-of-the-art (SoTA) designs and industry-standard models (VGG-16, ResNet-50, MobileViT).

4. Results and Performance Metrics

Hardware Efficiency (FPGA & ASIC)

• Resource Utilization (FPGA): Compared to SoTA designs (e.g., ReAFM, AxSF), the NEURIC design reduces LUTs by up to 4.5× and Flip-Flops (FFs) by up to 3.2×.
• Power Consumption: Achieves a 4.2× to 5.4× reduction in power compared to leading SoTA designs.
• ASIC Area: At 45nm and 28nm nodes, the design offers an area reduction of up to 16.2× compared to Zhang et al. [20] and 4× compared to RECON [11].
• Delay: Critical delay is improved by 30% compared to AxSF [16] and reduced to 2.56 µs (FPGA) and 2.6 ns (ASIC).

Accuracy and Quality of Results (QoR)

• Error Metrics: The Mean Error (ME) and Mean Absolute Error (MAE) are significantly lower than PWL and LUT-based approaches.
• Application Accuracy: When deployed in neural networks (VGG-16, ResNet-50, MobileViT), the design achieves a Quality of Results (QoR) of 98.5% compared to full floating-point baselines.
• Throughput: The 64-unit NEURIC Vector Engine achieves 52.3 GOPS with an inference time of 148ms, outperforming posit-based and fixed-point layer-reused architectures.
• Energy Efficiency: In Transformer workloads (MobileViT), the design improves energy efficiency by 2.5× over TRETA and 17.54× over SIGMA, while reducing power consumption from ~22W to ~1.54W.

5. Significance

This paper marks a pivotal shift in AI hardware design by demonstrating that reconfigurability does not require a trade-off in efficiency.

• Versatility: It solves the "one-size-fits-all" problem by allowing a single hardware block to adapt to the specific AF needs of DNNs, RNNs, and Transformers dynamically.
• Edge AI Viability: The drastic reduction in area and power makes this architecture ideal for resource-constrained edge devices and battery-powered AI applications.
• Future-Proofing: By supporting emerging activation functions (Swish, GeLU) alongside traditional ones, the NEURIC engine is positioned to support the evolving landscape of AI models without requiring hardware redesigns.
• Commercial Impact: The work validates the CORDIC approach as a superior alternative to LUTs and pseudo-CORDIC software implementations, influencing commercial IP blocks (e.g., AMD Xilinx, Intel DSP Builder) and offering a blueprint for next-generation AI accelerators.