Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review

This paper reviews the landscape of ultra-low-power edge and in-sensor AI processors and empirically benchmarks a segmentation model on GAP9, STM32N6, and Sony IMX500 platforms to demonstrate that while in-sensor processing offers superior energy-delay performance, different architectures provide distinct trade-offs between latency, energy efficiency, and power budgets.

The Gist

Imagine you have a tiny, battery-powered robot that needs to "see" and understand the world around it—like a smart doorbell that recognizes faces or a drone that avoids trees. In the old days, this robot would take a photo, send it all the way to a giant, super-fast computer in the cloud (like a massive data center), wait for the answer, and then act.

But that takes too long, uses too much battery, and isn't very private. Edge AI is the new trend: putting the "brain" directly inside the robot's tiny chip so it can think for itself instantly.

This paper is a race report comparing three different types of "brains" designed for these tiny robots. The authors wanted to see which one is the best at solving a specific, tricky puzzle: segmentation.

The Challenge: The "Pixel Painter"

To test these chips, the researchers didn't just ask them to guess "Is that a cat?" (which is easy). They asked them to do segmentation.

Think of it like this:

• Classification is like looking at a painting and saying, "That's a sunset."
• Segmentation is like taking a paintbrush and carefully coloring every single pixel in the painting to outline exactly where the sun ends and the clouds begin.

This is a much harder job. It requires the chip to remember a huge amount of detail while it works, which is like trying to juggle 100 balls while running a marathon. This "stress test" reveals which chips are truly efficient and which ones get tired (run out of battery) or drop the balls (get slow).

The Three Contenders

The paper puts three very different athletes in the ring:

1. The GAP9: The Marathon Runner (The Multi-Core Workhorse)

• What it is: A chip with many small, simple cores working together (like a team of 8 cyclists riding in a pack).
• The Analogy: Imagine a team of 8 people in a small room, all passing notes to each other to solve a problem. They are very energy-efficient and can keep going for a long time on a single AA battery.
• The Result: They are slow at finishing the race (high latency), but they use very little energy to do it. They are great if you need your robot to last for weeks on a tiny battery, even if it's not the fastest.

2. The STM32N6: The Sprinter (The Heavyweight Champ)

• What it is: A powerful chip with a specialized "neural engine" built in, designed for speed.
• The Analogy: Think of a Formula 1 car. It has a massive engine and goes incredibly fast. But, it drinks fuel like there's no tomorrow.
• The Result: It finished the race the fastest (lowest latency). However, it burned through its battery so quickly that it's not practical for devices that need to run for years on a coin cell. It's the "speed over everything" option.

3. The Sony IMX500: The Magician (The In-Sensor Prodigy)

• What it is: This is the most unique one. Instead of a separate brain, the "thinking" happens inside the camera sensor itself.
• The Analogy: Imagine a photographer who doesn't just take a picture and send it to a lab. Instead, the camera lens itself has a tiny brain that looks at the photo, figures out what's in it, and only sends you a text message saying "I see a dog." It never sends the whole photo.
• The Result: This was the winner. Because it doesn't have to move data around (no heavy lifting), it was incredibly fast and incredibly efficient. It used the least amount of energy and finished the race almost as fast as the heavy Formula 1 car.

The Big Takeaway

The paper concludes that there is no single "perfect" chip. It depends on what you need:

• Need it to last 5 years on a tiny battery? Go with the GAP9 (the Marathon Runner).
• Need it to react instantly and don't care about battery life? Go with the STM32N6 (the Sprinter).
• Need the best balance of speed and battery life? The Sony IMX500 (the Magician) is the future. It shows that the smartest way to build these devices is to stop moving data around and start thinking right where the data is born.

In short: The future of smart devices isn't just about making chips faster; it's about making them smarter about where they do the thinking. The Sony chip proved that doing the math right inside the camera is the magic trick that saves the most energy.

Technical Summary

Here is a detailed technical summary of the paper "Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review."

1. Problem Statement

The rapid migration of Artificial Intelligence (AI) workloads from cloud servers to edge devices (IoT, wearables, micro-robots) is constrained by strict energy, thermal, and latency budgets. Most edge systems operate under a 200 mW power ceiling, driven by battery limitations and the lack of active cooling.

• The Bottleneck: Traditional Microcontroller Units (MCUs) (e.g., ARM Cortex-M4/M33) lack the throughput for sustained Deep Neural Network (DNN) inference. Meanwhile, general-purpose processors and high-power accelerators exceed the thermal and energy limits of battery-operated devices.
• The Challenge: There is a lack of unified, empirical data comparing heterogeneous low-power architectures (MCUs with accelerators vs. in-sensor processors) under realistic workloads. Designers struggle to balance latency (responsiveness) against energy efficiency (operations per Joule) and Energy-Delay Product (EDP).

2. Methodology

The authors employ a two-pronged approach: a comprehensive architectural taxonomy and a controlled empirical benchmark.

A. Architectural Taxonomy

The paper categorizes ultra-low-power edge processors into distinct paradigms:

1. MCUs with ML Extensions: Standard cores (ARM Cortex-M55/M85, RISC-V) with DSP or vector extensions (e.g., Ambiq, STM32U5).
2. MCUs with Dedicated Accelerators: MCUs paired with Neural Processing Units (NPUs) or ConvNet engines (e.g., STM32N6, Max78000, GAP9).
3. Reconfigurable Logic: FPGAs offering custom datapaths (e.g., Lattice iCE40).
4. Neuromorphic Processors: Event-driven Spiking Neural Networks (SNNs) (e.g., Intel Loihi, IBM TrueNorth).
5. Processing/Compute-in-Memory (PIM/CIM): Performing MAC operations inside memory arrays to reduce data movement (e.g., Mythic, Analog CIM).
6. In-Sensor Computing: Integrating compute logic directly into the sensor readout pipeline (e.g., Sony IMX500).

B. Empirical Benchmarking

To move beyond theoretical peak performance (TOPS), the authors benchmarked a single, representative workload across three heterogeneous platforms:

• Workload: PicoSAM2, a 336 million Multiply-Accumulate (MAC) segmentation model (U-Net derived). Segmentation was chosen as a "stress test" because it maintains large activation tensors and high memory bandwidth requirements, exposing dataflow bottlenecks better than classification tasks.
• Platforms Tested:
  1. GAP9: A scalable RISC-V manycore cluster with hardware accelerators (MCU-class).
  2. STM32N6: An ARM Cortex-M55 core paired with a dedicated Neural Architecture Accelerator (NPU).
  3. Sony IMX500: A stacked CMOS sensor with integrated in-sensor DSP and NPU.
• Metrics Evaluated:
  • Latency: End-to-end inference time (ms).
  • Inference Efficiency: MACs per cycle (utilization of theoretical throughput).
  • Energy Efficiency: MACs per Joule (MMAC/J).
  • Energy-Delay Product (EDP): A composite metric balancing speed and energy cost.

3. Key Contributions

• Unified Comparative Framework: The paper provides a rare, apples-to-apples comparison of three distinct architectural paradigms (Manycore MCU, NPU-accelerated MCU, and In-Sensor) running the exact same complex model.
• Workload Selection: By using a segmentation model rather than simple classification, the study rigorously tests memory hierarchies and data movement efficiency, which are critical bottlenecks in ultra-low-power systems.
• Quantitative Trade-off Analysis: The study moves beyond "TOPS" to reveal practical metrics like MAC/cycle utilization and EDP, offering actionable insights for system designers.

4. Key Results

The benchmarking revealed significant divergences in hardware behavior:

Metric	GAP9 (RISC-V Manycore)	STM32N6 (ARM + NPU)	Sony IMX500 (In-Sensor)
Latency	42.13 ms (Slowest)	13.71 ms (Fastest)	14.30 ms
Clock Frequency	370 MHz	800 MHz	262.5 MHz
Inference Efficiency	20.77 MAC/cycle	29.52 MAC/cycle	86.24 MAC/cycle
Energy Efficiency	182.15 MMAC/J	21.47 MMAC/J	1359.65 MMAC/J
Energy-Delay Product	74.88 mJ·s	206.76 mJ·s	3.40 mJ·s

Detailed Findings:

• In-Sensor Dominance (IMX500): Despite having the lowest clock frequency, the IMX500 achieved the highest utilization (86.2 MAC/cycle) and lowest EDP (3.4 mJ·s). Its stacked CMOS architecture drastically reduces data movement overhead, resulting in an energy efficiency ~7.5x better than GAP9 and ~63x better than STM32N6.
• Latency vs. Power (STM32N6): The STM32N6 offered the lowest raw latency (13.7 ms) due to its high clock speed and dedicated NPU. However, this came at a massive energy cost, resulting in the worst Energy Efficiency and highest EDP.
• MCU Efficiency (GAP9): GAP9 offered the best energy efficiency among the MCU-class devices (182 MMAC/J) but suffered from higher latency due to lower clock speeds and memory bottlenecks inherent in the manycore architecture for this specific workload.

5. Significance and Conclusion

• No "One-Size-Fits-All": The paper concludes that no single architecture is optimal for all edge AI scenarios.
  • In-Sensor (IMX500) is superior for always-on, battery-constrained, and latency-sensitive applications where minimizing data transfer is paramount.
  • NPU-Accelerated MCUs (STM32N6) are suitable for applications where raw speed is the priority and power is less constrained.
  • Manycore MCUs (GAP9) remain competitive for flexible, low-power deployments where the cost of specialized in-sensor hardware is prohibitive.
• Design Implications: The results highlight that data movement is the primary energy consumer in edge AI. Architectures that minimize data transfer (like in-sensor processing) offer orders-of-magnitude improvements in efficiency compared to traditional Von Neumann architectures, even if the latter have higher clock speeds.
• Future Direction: The study validates the technological maturity of in-sensor processing as a critical pathway for the next generation of ultra-low-power, context-aware intelligent systems.