CORVET: A CORDIC-Powered, Resource-Frugal Mixed-Precision Vector Processing Engine for High-Throughput AIoT applications

The paper presents CORVET, a resource-frugal, runtime-adaptive vector processing engine that leverages CORDIC-based mixed-precision MAC units and time-multiplexed execution to achieve high throughput and energy efficiency for edge AI applications, demonstrating significant improvements in compute density and power savings over state-of-the-art solutions.

The Gist

Imagine you are running a busy, high-tech bakery (an AI system) that needs to bake thousands of loaves of bread (process data) every minute. In a traditional bakery, you have a massive, expensive oven that can bake a perfect loaf in one second, but it costs a fortune to run and takes up half the kitchen.

Now, imagine a new, smarter bakery manager (the CORVET engine) who changes the rules. Instead of one giant oven, they use a team of small, efficient workers who can bake bread in a few different ways depending on how important the loaf is.

Here is the story of this new engine, broken down into simple concepts:

1. The Problem: The "All-or-Nothing" Kitchen

In most current AI chips (the ovens), every calculation is treated the same. Whether you are baking a fancy wedding cake (a critical part of a self-driving car's vision) or a simple slice of toast (a background task), the machine uses the same heavy, energy-hungry process.

• The Waste: It's like using a sledgehammer to crack a nut. The machine wastes energy and space on tasks that don't need such precision.
• The Bottleneck: A lot of the kitchen space is dedicated to "decorating" the bread (Activation Functions), but these decorators often sit idle while the bakers are working. This leaves a lot of "dark silicon" (empty, unused space on the chip).

2. The Solution: The "Smart, Adaptable" Team (CORVET)

The authors built a new engine called CORVET. Think of it as a team of versatile workers who can switch hats instantly.

A. The "CORDIC" Math Trick (The Swiss Army Knife)

Instead of using a heavy, specialized calculator for every math problem, CORVET uses a clever, lightweight tool called CORDIC.

• The Analogy: Imagine a carpenter who usually uses a heavy power saw. But for small, quick cuts, they switch to a simple, foldable pocket knife. The pocket knife takes a few more seconds to make the cut, but it's tiny, cheap, and uses almost no electricity.
• How it works: CORVET uses this "pocket knife" method (iterative CORDIC) to do math. It can choose to be fast and rough (approximate mode) for simple tasks, or slow and precise (accurate mode) for critical tasks. It switches between these modes instantly while the machine is running.

B. The "Time-Sharing" Decorators (Multi-Activation Block)

In the old kitchens, every worker had their own dedicated decoration station. If they weren't decorating, the station sat empty.

• The New Way: CORVET has one super-efficient decoration station that everyone shares.
• The Analogy: Think of a single chef who can flip between making a salad, grilling a steak, or baking a pie, depending on who is hungry right now. By sharing this station, the kitchen saves massive amounts of space and energy. The chef is always busy, so there is no "dark silicon" (wasted space).

C. The "Teamwork" Strategy (Vector Processing)

Since the "pocket knife" math takes a tiny bit longer per calculation, how does the machine stay fast?

• The Analogy: If one worker takes 3 seconds to cut a loaf, but you have 256 workers doing it at the same time, you get 256 loaves in 3 seconds.
• The Result: The engine uses parallel lanes. Even though individual math steps are slightly slower, the sheer number of workers working together makes the whole system incredibly fast and efficient.

3. The Results: A Smarter, Leaner Bakery

When the researchers tested this new engine:

• Energy Savings: It uses significantly less power (like switching from a gas-guzzling truck to an electric scooter).
• Speed: It can process data up to 4 times faster than previous designs using the same amount of hardware.
• Density: It packs more computing power into a smaller space (like fitting a supercomputer into a shoebox).
• Real-World Test: They put it on a small board (Pynq-Z2) and used it to make a drone recognize objects. It worked faster and used less battery than expensive commercial chips like the NVIDIA Jetson Nano.

Summary

CORVET is like a chameleon for computer chips. It doesn't force every task to be perfect and expensive. Instead, it asks, "How perfect does this need to be?"

• If it's a rough guess? It uses the fast, cheap, low-power mode.
• If it's life-or-death (like avoiding a pedestrian)? It switches to the slow, precise mode.

By being flexible, sharing resources, and working in a team, it solves the biggest problem in Edge AI: How to make smart computers that fit in your pocket without draining your battery.

Technical Summary

1. Problem Statement

Deep learning inference on resource-constrained edge devices (AIoT) faces significant challenges regarding energy efficiency, area, and latency.

• Computational Bottlenecks: Multiply-Accumulate (MAC) operations constitute ~90% of Deep Neural Network (DNN) workloads, while Non-Linear Activation Functions (NAFs) account for 2–5%.
• Inefficiencies in Current Accelerators:
  • Fixed Approximation: Existing approximate computing designs often use fixed approximation points, leading to irreversible accuracy degradation or requiring error-compensation logic that negates energy savings. They lack the flexibility to adapt to the varying sensitivity of different network layers.
  • Hardware Underutilization: Dedicated hardware blocks for activation functions (AFs) often remain idle for significant portions of execution (up to 84% idle cycles in some architectures), leading to "dark silicon" and poor area efficiency.
  • Rigidity: Many designs enforce static precision and fixed datapaths, making them suboptimal for heterogeneous workloads requiring mixed-precision computation.

2. Methodology and Architecture

The authors propose CORVET, a runtime-adaptive vector engine designed to balance accuracy, latency, and resource usage through a hardware-software co-design approach.

A. Core Architecture

• Iterative CORDIC-Based MAC Unit: Instead of using pipelined or fixed-stage CORDIC units, CORVET employs an iterative MAC architecture.
  • Mechanism: It reuses a single CORDIC datapath across multiple clock cycles.
  • Adaptability: The number of active iterations is configurable per layer. Layers sensitive to numerical error run in "accurate mode" (more iterations), while less sensitive layers run in "approximate mode" (fewer iterations).
  • Precision Support: Supports flexible precision (4/8/16-bit) and dynamic switching between approximate and accurate modes without structural changes or auxiliary correction logic.
• Time-Multiplexed Multi-Activation-Function (Multi-AF) Block:
  • To address the underutilization of AF hardware, a single time-multiplexed block is shared across all Processing Elements (PEs).
  • It supports a wide range of functions (Sigmoid, Tanh, SoftMax, GELU, Swish, ReLU, SELU) using common CORDIC resources.
  • This design achieves high utilization (72–86%) with minimal area overhead (<4% increase).
• Vector Engine Organization:
  • Composed of $N$ homogeneous PEs (scalable from 64 to 256).
  • Uses a lane-based execution model where iterative MAC latency is amortized across parallel lanes, enabling high throughput without deep pipelining.
  • Includes a lightweight control engine, data prefetcher, and Absolute Average Deviation (AAD) pooling/normalization units.

B. Control and Data Flow

• Runtime Adaptation: A control engine manages configuration registers to set precision, iteration count, and execution sequencing on a per-layer basis.
• Memory Mapping: Uses a flexible addressing scheme for weights and biases to avoid unused address locations, supporting scalable DNN topologies.
• Data Flow: Follows a streaming execution model where input feature maps are fetched, buffered, and broadcast to the vector engine. Partial sums are forwarded to subsequent MAC stages or the AF pipeline.

3. Key Contributions

1. Runtime-Configurable MAC: A low-resource, iterative CORDIC MAC unit that allows dynamic trade-offs between accuracy and latency at the layer level.
2. Scalable Vector Engine: An architecture that amortizes iterative latency across parallel lanes, achieving up to 4× throughput improvement within the same hardware resources compared to fixed-stage designs.
3. High-Utilization Multi-AF Block: A time-multiplexed block supporting diverse nonlinear functions, significantly reducing "dark silicon" and hardware overhead.
4. Comprehensive Evaluation: A full-stack validation spanning software emulation, FPGA prototyping, and 28 nm ASIC synthesis, demonstrating system-level gains on CNN and Transformer workloads.

4. Experimental Results

The design was evaluated using Python emulation, FPGA (AMD Virtex-7 VC707), and ASIC (28 nm HPC+ CMOS, 0.9V) synthesis.

A. Circuit-Level Efficiency (ASIC 28nm)

• MAC Performance: Each MAC stage achieves a 33% reduction in time and 21% reduction in power compared to state-of-the-art (SoTA) CORDIC-based designs.
• Area: The iterative approach significantly reduces area compared to pipelined alternatives by reusing datapaths.

B. System-Level Performance

• Compute Density & Energy Efficiency:
  • The 256-PE configuration achieves a peak compute density of 4.83 TOPS/mm².
  • Energy efficiency reaches 11.67 TOPS/W.
• FPGA Implementation:
  • Operating at 85.4 MHz on VC707, it delivers 6.43 GOPS/W at only 0.53 W power consumption, outperforming several SoTA designs that rely on DSP blocks or higher frequencies.
• End-to-End Deployment (Pynq-Z2):
  • For object detection and classification (VGG-16), the system achieved a total latency of 84.6 ms at 0.43 W.
  • This outperforms commercial embedded platforms like the NVIDIA Jetson Nano (226 ms / 1.34 W) and Raspberry Pi (555 ms / 2.7 W).

C. Accuracy

• Approximate Mode: Incurs ~2% accuracy degradation at the application level.
• Accurate Mode: Limits accuracy loss to <0.5%.
• Heuristic Selection: By applying an accuracy-sensitivity heuristic to select iteration depth per layer, the system retains most performance benefits of approximate execution while preserving end-to-end model accuracy without retraining.

5. Significance and Conclusion

CORVET represents a significant advancement in edge AI hardware by bridging the gap between fixed-approximation accelerators and resource-intensive accurate designs.

• Flexibility: It solves the rigidity of prior designs by enabling fine-grained, runtime control over numerical fidelity.
• Efficiency: By eliminating "dark silicon" through time-multiplexed activation functions and reducing MAC complexity via iteration, it offers superior energy efficiency and area utilization.
• Scalability: The modular vector engine design allows seamless scaling from small embedded devices to larger edge servers without redesigning the core datapath.

The paper concludes that CORVET provides a scalable, energy-efficient compute substrate ideal for next-generation AIoT applications, with future work focusing on compiler-assisted automation for precision selection and integration with RISC-V SoCs.