🧬 biology

Entropy-driven adaptive neural architecture search for cell segmentation on edge devices

This paper proposes a hardware-aware, entropy-driven neural architecture search method based on the U-Net backbone that optimizes cell segmentation models for resource-constrained edge devices, achieving competitive accuracy across diverse datasets while ensuring efficient deployment.

Original authors: Yiwen Xu, Tariq M. Khan, Yanming Zhu, Yang Song, Erik Meijering

Published 2026-10-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiwen Xu, Tariq M. Khan, Yanming Zhu, Yang Song, Erik Meijering

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

In the quiet hum of a laboratory, scientists often watch cells move, divide, and interact through the lens of a microscope. These microscopic events happen in real time, shifting and changing in ways that reveal how life grows and heals. To understand these processes, researchers must capture thousands of images and then carefully trace the outline of every single cell, a task that is both tedious and critical. For decades, this work relied on powerful computers sitting in server rooms, far away from the microscope. But a new frontier has emerged: the idea of placing the computing power directly next to the microscope, on small, portable devices that can watch cells continuously without needing a massive data center. This shift promises to let scientists intervene instantly when a cell behaves unexpectedly, turning passive observation into active discovery. However, these small devices have strict limits on their memory and battery life, making it difficult to run the complex software usually required to see cells clearly.

A team of researchers at the University of New South Wales and Griffith University has developed a new way to solve this problem. They created a method that automatically designs a custom computer program for each specific type of small device, ensuring it is powerful enough to see cells but light enough to run on limited battery power. Instead of trying to force a single, heavy-duty program to work on every device, their system builds a unique, streamlined architecture for each one. The researchers tested this approach on five different types of small computing boards, ranging from a Raspberry Pi to specialized medical imaging hardware. They found that their method could identify cell boundaries with high accuracy across ten different types of cell images, including those taken with various lighting techniques like fluorescence and bright-field microscopy. The resulting programs were not only accurate but also fast enough to process images in real time, using significantly less memory and energy than previous methods.

The core of this achievement lies in how the researchers taught the computer to design itself. Traditionally, creating a program to find cells involves a human expert manually selecting and arranging different mathematical tools, a process that takes weeks and requires deep expertise. The new method, known as neural architecture search, automates this design process. It starts with a basic blueprint for a cell-finding program and then explores millions of possible variations to find the best one. The challenge has always been that the best-performing programs are often too large and slow for small devices. The researchers solved this by adding a set of rules that act like a strict budget during the design phase. The system is told that it cannot exceed a certain amount of time to process an image or a certain amount of memory. If a design tries to use too much of either, the system penalizes it, forcing the search to find simpler, more efficient solutions.

To make this work without sacrificing accuracy, the team introduced a clever balancing act. They found that simply punishing complex designs often led to programs that were too simple to see the cells clearly. To fix this, they added a mechanism that rewards the program for keeping its internal "thinking" diverse and rich. Imagine the program as a team of workers; if the budget cuts are too severe, the team might fire everyone and leave the work undone. The researchers' method ensures that even with a tight budget, the remaining workers are assigned tasks that maximize their unique value, ensuring the final picture is still sharp and detailed. This approach allowed the system to discover networks that were highly efficient, using only the necessary components to get the job done.

The results were tested on a wide variety of cell types, from stem cells to cancer cells, and across different imaging styles. The custom-designed programs performed as well as, and often better than, existing methods that were not designed with these small devices in mind. On the smallest devices, the new programs processed images in under 100 milliseconds, a speed fast enough to keep up with the rapid changes happening inside a living cell. They also used significantly less memory, with some models fitting into just 10 megabytes of storage, compared to the hundreds of megabytes required by older, heavier models. This efficiency means that a scientist could potentially leave a small device running in an incubator for days, watching cells divide and move, with the device analyzing the data instantly and alerting the researcher only when something important happens.

The researchers also verified that their method of predicting how fast a program would run was accurate. They compared their computer's estimates against the actual time it took for the programs to run on the physical hardware. The estimates were very close to reality, with the predicted times differing from the actual times by less than 11 percent on most devices. This reliability is crucial because it means scientists can trust the system to find the right design without having to test every single possibility on the actual hardware, which would be too slow and expensive. By proving that these lightweight, custom-built programs can handle the complexity of real biological data, the study opens the door to a new era of portable, real-time biological research.

This work suggests that the future of cell observation may not depend on massive supercomputers, but on small, intelligent devices that can think for themselves. The researchers demonstrated that by carefully balancing the need for speed and memory against the need for accuracy, it is possible to create tools that are both powerful and practical. Their method does not just make existing technology faster; it fundamentally changes how these tools are created, allowing them to adapt to the specific constraints of the device they will run on. As these small devices become more common in laboratories, this approach could allow for a level of continuous, real-time monitoring that was previously impossible, giving scientists a clearer, more immediate view into the dynamic world of the cell.

Technical Summary: Entropy-driven Adaptive Neural Architecture Search for Cell Segmentation on Edge Devices

Problem Statement
Accurate, real-time segmentation of cells in microscopy images is essential for biomedical research, particularly for monitoring dynamic cellular processes like migration and proliferation. While deep learning has improved segmentation accuracy, deploying these models on edge devices (e.g., incubator microscopes) presents significant challenges. Edge devices have limited computational power, memory, and energy resources. Furthermore, manually designing optimal network architectures for diverse cell types and staining methods is time-consuming and requires expertise. Existing Neural Architecture Search (NAS) methods often fail to account for specific hardware constraints, resulting in architectures that are too complex or memory-intensive for edge deployment. Additionally, standard hardware-aware NAS methods are often tailored for image classification and lack the multiscale feature extraction capabilities required for high-resolution microscopy segmentation.

Methodology
The authors propose a hardware-aware NAS framework based on the U-Net backbone, specifically designed for live-cell segmentation on resource-constrained devices. The method consists of three core components:

Search Space Construction:
- Topology: The framework utilizes a U-Net structure with encoder-decoder pathways. Unlike previous methods that use repetitive blocks, this approach constructs the network with non-repetitive building blocks, allowing Basic Operations (BOs) to differ within each block for flexible feature capture.
- Operations: The search space includes a curated set of BOs (Convolution, Atrous Convolution, Shuffle Convolution, Depthwise-separable Convolution, Squeeze-and-Excitation, Identity, and "None"). The "None" operator allows nodes to be bypassed, enabling structural sparsity.
- Pruning: Upsampling and downsampling operations (Max Pooling and Transposed Convolution) are fixed to reduce the search space, while the search focuses on the internal composition of encoder/decoder units.
Hardware-Aware Search Strategy:
- Multi-Objective Optimization: The search minimizes a composite loss function: $L_{total} = w_1 \cdot L_{ce} + w_2 \cdot \text{Latency} + w_3 \cdot \text{Memory} - w_4 \cdot H$ .
- Latency and Memory Estimation: To avoid the prohibitive cost of running every candidate architecture on target hardware, the method employs device-specific Look-Up Tables (LUTs). These LUTs store the latency and memory usage of individual BOs measured on specific edge devices (Jetson Nano, PYNQ-Z2, ZUBoard-1CG, Raspberry Pi 5, and Raspberry Pi 5 with HAILO-8).
- Soft Constraints: Instead of hard cutoffs that discard architectures slightly exceeding thresholds, the method applies quadratic penalty terms for violations of latency ( $T_{latency}$ ) and memory ( $T_{memory}$ ) limits. This allows the search to explore near-feasible solutions while discouraging infeasible ones.
Entropy-Driven Optimization:
- To counteract the potential loss of segmentation accuracy caused by aggressive hardware constraints (which may lead to overly sparse networks), the authors introduce an entropy-driven mechanism.
- Information entropy ( $H$ ) is calculated based on the differential entropy of feature map activations (assuming a Gaussian distribution).
- Maximizing entropy serves as a reward term, encouraging the network to maintain diverse and informative feature representations even when constrained by hardware limits. This prevents the model from collapsing into overly simple configurations that degrade performance.

Key Contributions

Hardware-Aware NAS for Microscopy: The paper presents a NAS method specifically tailored for cell segmentation, addressing the multiscale feature requirements of microscopy images which generic hardware-aware NAS methods often overlook.
Entropy-Driven Mechanism: The introduction of an entropy-based regularization term to balance hardware constraints with feature representation capacity is a novel contribution, ensuring that efficiency gains do not come at the cost of critical segmentation accuracy.
Non-Repetitive U-Net Architecture: The framework moves away from repetitive block designs, allowing for flexible, adaptive architectures where operations vary across different scales.
Comprehensive Evaluation: The method is evaluated across five distinct edge hardware platforms and ten diverse 2D live-cell imaging datasets from the Cell Tracking Challenge (CTC), covering various modalities (Bright-field, DIC, Fluorescence, Phase Contrast) and cell types.

Results

Segmentation Performance: On the CTC datasets, the proposed method achieved competitive state-of-the-art performance in SEG (overlap) and DET (detection) metrics. For instance, on the BF-C2DL-HSC subset, it achieved a SEG score of 0.825 and DET of 0.990, outperforming or matching other NAS-based models like NAS-UNet, BiX-NAS, and Auto-DeepLab.
Hardware Efficiency: The proposed models demonstrated significantly lower latency and memory usage compared to standard models like nnU-Net and AdwU-Net. On the Jetson Nano, the method achieved inference times around 58ms with a model size of only 10 MB, whereas nnU-Net required 278ms and 37 MB.
Ablation Studies:
- Hardware Penalties: The study confirmed that hardware-aware penalties effectively prune superfluous parameters and increase the usage of the "None" operator, reducing model size without catastrophic accuracy loss.
- Entropy Optimization: The entropy-driven term was shown to accelerate convergence during the search phase and improve final SEG performance. It successfully offset the representational capacity loss caused by hardware constraints, yielding a model with better accuracy than the same architecture without entropy optimization, while maintaining similar model size.
LUT Validation: The LUT-based latency estimator showed high consistency with actual measured hardware latency (Spearman's $\rho > 0.94$ across devices), validating its use as an efficient proxy during the search process.

Significance and Claims
The authors claim that their framework successfully bridges the gap between high-accuracy cell segmentation and the strict resource limitations of edge devices. By integrating hardware constraints directly into the search process and using entropy to preserve feature diversity, the method produces architectures that are both efficient and accurate. The paper positions this work as a solution for real-time, distributed monitoring of cellular activities in incubator-like environments, where reliance on desktop workstations or servers is impractical due to bandwidth and computational constraints. The study emphasizes that the proposed approach achieves a favorable trade-off, enabling reliable, real-time cell image analysis on diverse, low-power hardware platforms without requiring manual architecture design.

More like this