💻 computer science

Quantization Trade-offs in Edge AI: Evaluating Lightweight Neural Architectures for Mobile-Based Healthcare and Agricultural Diagnostics

This study empirically evaluates lightweight neural architectures and compression techniques on ARM-based edge devices, demonstrating that MobileNetV2 with INT8 quantization offers the optimal balance of accuracy and efficiency for mobile healthcare and agricultural diagnostics, while highlighting the severe accuracy trade-offs of pruning on complex visual data.

Original authors: NUR ALAM RAKIB, MD NAHIDUL ISLAM

Published 2026-11-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: NUR ALAM RAKIB, MD NAHIDUL ISLAM

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where a doctor in a remote village or a farmer in a vast, sun-drenched field can diagnose a serious illness or a crop disease instantly, without needing to send samples to a distant laboratory or wait for a cloud server to process the data. This is the promise of edge artificial intelligence: the ability to run sophisticated computer vision directly on small, battery-powered devices like smartphones or handheld scanners. For this to work, the software must be incredibly efficient, fitting into tiny amounts of memory and running fast enough to give an answer in seconds, all while preserving the privacy of the user's data. The challenge lies in the fact that the most accurate computer models for seeing and diagnosing are usually massive, requiring powerful computers that simply do not exist in these off-grid settings. Researchers have long sought a way to shrink these giant models down to fit on a pocket device without losing their ability to see the subtle details that separate a healthy leaf from a diseased one, or a benign skin spot from a dangerous tumor.

A team of researchers at Nanjing University of Information Science and Technology set out to solve this specific puzzle by testing how far they could compress these digital brains before they broke. They focused on two critical areas: mobile healthcare, where devices screen for skin cancer and malaria parasites, and precision agriculture, where cameras identify diseases on mango and soybean leaves. The team took five different types of lightweight neural networks—software architectures designed to be small and fast—and tested them on five different real-world datasets. These datasets ranged from high-resolution images of skin lesions and microscopic blood cells to photos of crop leaves taken in messy, unpredictable field conditions. The researchers wanted to see which of these small models could survive the harsh reality of being shrunk down for a mobile device without losing their diagnostic accuracy.

To make these models fit on a device like a Raspberry Pi 5, a small single-board computer often used as a testbed for edge technology, the team applied two different methods of compression. The first method was like taking a high-resolution photograph and converting it to a lower-quality format to save space; in technical terms, this is called quantization. They converted the model's internal numbers from a high-precision format that takes up a lot of memory into a simpler, 8-bit integer format. The second method was more drastic: they attempted to cut out entire sections of the model's brain that they thought were redundant, a process known as structured pruning. They then ran these compressed models on the hardware to measure how fast they could make a diagnosis, how much memory they used, and, most importantly, whether they could still get the right answer.

The results revealed a clear and surprising truth about how to build these diagnostic tools. The method that simply reduced the precision of the numbers, known as dynamic 8-bit quantization, worked remarkably well. It cut the size of the models in half, reducing a model that took up nearly 9 megabytes of storage to just over 4 megabytes, yet the model's ability to diagnose remained almost exactly the same. For instance, a model used to screen for skin lesions maintained an accuracy of 92.40% after compression, virtually identical to its original performance. Similarly, models analyzing crop diseases held their ground, with one maintaining 90.00% accuracy on complex field data. This approach proved to be a safe and reliable way to shrink the software without damaging its intelligence.

In stark contrast, the method that tried to physically remove parts of the model's structure, known as structured pruning, proved to be a dangerous gamble. While it did reduce the number of active connections, it caused the models to lose their ability to recognize fine details. On complex datasets like the PlantDoc collection, which features crops photographed in real-world fields with shadows and clutter, pruning caused accuracy to plummet by as much as 25%. The models became confused, failing to distinguish between healthy and diseased leaves or between different types of skin conditions. The researchers found that this aggressive cutting stripped away the very features the models needed to see the subtle patterns that define a diagnosis, rendering them unreliable for real-world use.

The study also highlighted that not all small models are created equal. One architecture, called MobileNetV2, emerged as the most balanced choice, offering the best mix of high accuracy and resilience to compression. It performed consistently well across both medical and agricultural tasks, making it a strong candidate for smartphones and handheld scanners. Another model, ShuffleNetV2, was the fastest, capable of making a decision in just 7 milliseconds, but it was fragile; it struggled when the input images were noisy or complex, and its performance dropped significantly when pruned. The researchers also tested newer, nano-scale object detectors adapted for classification, finding that while they were robust, they required more memory and processing power than the simpler models, sometimes exceeding the strict limits of low-power devices.

Ultimately, the work provides a clear roadmap for engineers and doctors who want to bring advanced diagnostics to the field. The findings suggest that the best path forward is to use dynamic quantization to shrink models, as it preserves their intelligence while saving space, rather than trying to prune them, which risks destroying their ability to see. By choosing the right model architecture and applying the correct compression technique, it is possible to deploy powerful diagnostic tools on devices that fit in a pocket, operate without internet access, and run on a single battery charge. This means that in the near future, a farmer could point a phone at a sick plant and get an immediate answer, or a community health worker could screen for malaria in a remote clinic, all powered by software that has been carefully tuned to be both small and smart.

Technical Summary: Quantization Trade-offs in Edge AI for Mobile Healthcare and Agricultural Diagnostics

1. Problem Statement

The deployment of deep learning models for real-time diagnostic decision-making in resource-constrained, off-grid environments—specifically mobile point-of-care (PoC) healthcare and precision agriculture—faces a critical intersection of competing constraints. While Convolutional Neural Networks (CNNs) have established state-of-the-art diagnostic standards, conventional high-capacity architectures (e.g., ResNet-50, VGG-16) require multi-gigabyte memory footprints and high-power GPU acceleration, rendering them impractical for edge devices such as ARM-based microcontrollers, handheld scanners, and low-cost smartphones.

Existing literature often evaluates lightweight architectures or compression techniques in isolation on single, well-curated datasets, failing to address the "domain isolation" problem where models must perform across both clinical (e.g., skin lesions, malaria pathology) and agricultural (e.g., crop disease) domains. Furthermore, there is a lack of empirical data regarding the specific trade-offs between predictive accuracy, inference latency, dynamic RAM utilization, and hardware energy when applying aggressive compression techniques like structured filter pruning versus post-training quantization on noisy, real-world diagnostic inputs.

2. Methodology

This study presents a rigorous empirical evaluation of model compression techniques applied to five lightweight neural architectures: MobileNetV2, MobileNetV3Small, ShuffleNetV2 x0.5, YOLOv8n, and YOLOv11n (adapted for image-level classification via Global Average Pooling).

2.1 Experimental Framework

Datasets: The evaluation spans five benchmark datasets across two domains:
- Healthcare: ISIC 2019 / HAM10000 (skin lesion screening) and NIH Malaria (blood smear cell pathology).
- Agriculture: PlantDoc (high field noise), Mango Leaf Disease, and Soybean Leaf Disease.
Compression Techniques:
- Post-Training Quantization (PTQ): Dynamic 8-bit Integer (INT8) quantization, converting weights and activations from 32-bit floating-point (FP32) to 8-bit integers without retraining.
- Structured Pruning: Magnitude-based L1-norm filter pruning, removing entire convolutional channels to reduce FLOPs and memory.
Hardware Platform: All models were converted to ONNX and TensorFlow Lite (TFLite) formats and profiled on a Raspberry Pi 5 (Quad-core ARM Cortex-A76, 8GB RAM) to measure real-time inference latency, peak RAM usage, and flash storage footprint.

2.2 Pipeline

The workflow followed a standardized 5-stage pipeline:

Training: Fine-tuning FP32 baselines with spatial data augmentation.
Compression: Execution of structured pruning and/or dynamic INT8 PTQ.
Export: Conversion to ONNX with BatchNorm fusion.
Quantization: Conversion to TFLite with dynamic INT8 scaling.
Profiling: Deployment and benchmarking on the edge hardware.

3. Key Contributions

Cross-Domain Benchmarking: A quantitative evaluation of lightweight backbones across diverse clinical and agricultural datasets, demonstrating that model performance varies significantly based on domain complexity (e.g., noisy field images vs. controlled dermoscopy).
Ablation of Compression Strategies: A systematic comparison revealing that Dynamic INT8 PTQ preserves baseline representational accuracy across domains, whereas Structured Pruning induces severe accuracy degradation (up to 25.15% drop) on complex, noisy diagnostic inputs.
Hardware-Aware Profiling: Empirical benchmarks of latency, RAM, and storage on a Raspberry Pi 5, identifying specific architectural bottlenecks (e.g., ShuffleNetV2's low latency vs. MobileNetV3's CPU latency due to h-swish activations).
Selection Matrix & Protocol: The establishment of a dual-domain architectural selection matrix and a standardized 4-stage edge compression deployment protocol tailored for low-power, connectivity-impaired environments.

4. Experimental Results

4.1 Accuracy and Compression Trade-offs

Quantization Resilience: Dynamic INT8 PTQ consistently reduced model storage footprints by approximately 50% (e.g., MobileNetV2 reduced from 9.0 MB to 4.2 MB) while maintaining baseline FP32 accuracy.
- MobileNetV2 on ISIC 2019: 92.45% (FP32) $\rightarrow$ 92.40% (INT8).
- MobileNetV2 on PlantDoc: 89.86% (FP32) $\rightarrow$ 90.00% (INT8).
- MobileNetV2 on NIH Malaria: 97.10% (FP32) $\rightarrow$ 97.10% (INT8).
Pruning Fragility: Magnitude-based structured pruning resulted in catastrophic performance drops, particularly on visually complex datasets.
- MobileNetV2 on PlantDoc: Dropped from 89.86% (FP32) to 67.14% (Pruned).
- ShuffleNetV2 x0.5 on PlantDoc: Dropped from 75.57% (FP32) to 64.71% (Pruned).
- The study notes that pruning removes essential spatial filters required for fine-grained features like pigment networks and intracellular parasites.

4.2 Hardware Profiling (Raspberry Pi 5)

ShuffleNetV2 x0.5: Achieved the lowest latency (7 ms) and minimal memory overhead (2.4 MB RAM, 1.5 MB Flash), making it ideal for ultra-low-power IoT sensors, though it remains sensitive to pruning.
MobileNetV2: Offered the optimal balance, with 46 ms inference time and 4.2 MB storage (INT8), maintaining high accuracy across both domains.
YOLOv8n/v11n: While robust for multi-scale spatial features, these models exhibited higher latency (28–31 ms) and exceeded strict RAM limits on the edge device.
MobileNetV3Small: Suffered from CPU latency bottlenecks (137 ms) due to h-swish activation functions, despite its compact size.

4.3 Feature Attention (Grad-CAM)

Visual analysis confirmed that models retaining high accuracy (MobileNetV2/V3 with INT8) maintained focused attention on diagnostic features (e.g., irregular pigment networks in melanoma, ring-form trophozoites in malaria), whereas pruned models lost the ability to localize these subtle features.

5. Significance and Conclusions

The paper concludes that Dynamic Post-Training Quantization (INT8 PTQ) is the superior compression strategy for Edge AI diagnostics, offering a ~50% reduction in storage without compromising diagnostic accuracy. Conversely, Structured Filter Pruning is deemed unsuitable for shallow architectures deployed on noisy, unconstrained diagnostic data due to its tendency to strip essential representational capacity.

The study establishes MobileNetV2 as the optimal backbone for general-purpose mobile diagnostics (balancing accuracy and latency) and ShuffleNetV2 x0.5 as the preferred choice for ultra-low-power, latency-critical microcontroller applications. The authors propose a standardized deployment protocol that prioritizes BatchNorm fusion and INT8 quantization while avoiding aggressive pruning, providing a actionable framework for real-world integration in off-grid healthcare and agricultural settings. Future work is suggested to explore Quantization-Aware Training (QAT) for sub-8-bit precision and hardware-aware Neural Architecture Search (NAS).