💻 computer science

MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices

This paper introduces MiLSD, a micro line-segment detector optimized for resource-constrained devices that achieves a significant accuracy improvement on the ShanghaiTech Wireframe dataset by systematically evaluating output representations, quantization effects, and post-processing strategies within a strict sub-megabyte memory budget.

Original authors: Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand, M. Hassan Najafi

Published 2026-07-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand, M. Hassan Najafi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a tiny, battery-powered robot (like a smart doorbell or a toy car) to "see" straight lines in a room. In the world of computer vision, finding lines is crucial for tasks like mapping a room or checking if a machine part is built correctly.

The problem is that the "smart" AI models that are really good at this usually live on powerful computers (like GPUs) or smartphones. They are like giant libraries full of books (data) and require a massive mansion (memory) to store them. A tiny microcontroller (the robot's brain) is more like a pocket-sized notebook with only a few pages of space. It can't hold the giant library.

This paper introduces MiLSD, a new way to teach these tiny robots to find lines without needing a mansion. Here is how they did it, broken down into simple concepts:

1. The "Language" Problem (Output Representations)

Before teaching the robot, the researchers had to decide how to ask it to describe a line. They tried three different "languages":

The Heatmap (The "Foggy Map"): This method asks the robot to paint every single pixel that might be part of a line. It's like trying to draw a straight line by coloring in every dot on a grid. It's messy and requires a second step to connect the dots.
The Center + Displacement (The "Anchor and Rope"): This method finds the middle of the line and then asks, "How far and in what direction do the two ends go?" It's better, but the math gets tricky for a tiny brain.
The F-Clip (The "Compact ID Card"): This is the winner. Instead of guessing, the robot is taught to describe a line using just four facts: Where is the center? How long is it? And what angle is it?
- The Analogy: Imagine describing a stick to a friend. Instead of pointing at every inch of it (Heatmap) or giving complex coordinates, you just say, "It's here, it's 10 inches long, and it's tilted 45 degrees." This "ID Card" method is so efficient that even a tiny model with very few parameters (25,000) could learn it well.

2. The "Compression" Problem (Quantization)

To fit the model into the tiny notebook, the researchers had to shrink the numbers inside the AI.

Full Precision (fp32): Like measuring a table with a ruler that has marks down to the millimeter. Very accurate, but takes up a lot of space.
8-bit Quantization: Like using a ruler with marks every centimeter. The paper found that for line detection, this is almost as good as the millimeter ruler. The robot can still see the lines perfectly.
4-bit Quantization: Like using a ruler with marks only every 10 centimeters. The paper found this was a disaster. The robot got confused, especially when trying to guess the angle of the line. It's like trying to guess a precise angle with a very coarse ruler; the error is too big. Even with special training to help, the robot couldn't recover the lost accuracy.

3. The "Upgrade" Problem (Scaling Up)

The researchers started with the smallest possible robot (on an STM32F746 chip with 320KB of memory). It worked, but it was a bit slow and not super accurate.
Then, they moved to a slightly larger robot (STM32H7 with 1MB of memory). This is like upgrading from a pocket notebook to a small journal.

More Space = Smarter Brain: With this extra space, they could make the "brain" (the neural network) bigger. This alone improved the accuracy significantly.
The "Refinement" Tricks: They added three clever tricks that happen while the robot is looking at the image, without needing to retrain the brain:
1. Sub-pixel decoding: Instead of just saying "the line is here," the robot estimates the line is slightly to the left or right of the pixel grid, making it much sharper.
2. Test-Time Augmentation (TTA): The robot looks at the image, then looks at it upside down, sideways, and diagonally, and averages the results. It's like asking a group of friends to look at a blurry sign and averaging their guesses to get the right answer.
3. The "Verifier" (Line-of-Interest): The robot makes a list of potential lines, but some are fake (like shadows or texture). A tiny "judge" module looks at each candidate and says, "Yes, that's a real line" or "No, that's just noise." This step boosted the accuracy the most.

The Result

By combining the efficient "ID Card" language, the safe 8-bit compression, and the "Refinement" tricks, the researchers built MiLSD.

On the tiny robot, it found lines with a score of 10.6.
On the slightly larger robot with the extra tricks, it jumped to a score of 24.1.

The Bottom Line:
The paper proves that you don't need a giant supercomputer to find lines. If you choose the right way to describe the line (F-Clip), compress the numbers carefully (8-bit), and add a few smart "post-processing" tricks, you can get a very capable line detector that fits entirely inside a tiny, low-cost microcontroller chip. It's a bridge between "dumb" classical detectors and "smart" but heavy AI models.

Technical Summary: MiLSD – A Micro Line-Segment Detector for Resource-Constrained Devices

Problem Statement

Line segment detection is a fundamental primitive for visual SLAM, 3D reconstruction, and industrial inspection. While recent deep learning methods have achieved high accuracy, they typically require GPU-class memory (megabytes to gigabytes), rendering them incompatible with low-cost microcontrollers (MCUs) that often possess only hundreds of kilobytes of SRAM. Conversely, classical detectors (e.g., LSD, EDLines) are lightweight but suffer from degraded accuracy in low-contrast or cluttered environments and lack the static memory bounds required for deterministic embedded deployment. There is currently a gap in the literature for learned line-segment detectors that operate effectively within sub-megabyte memory budgets on general-purpose MCUs.

Methodology

The authors propose MiLSD, a detector tailored for MCU-level constraints, and conduct a systematic investigation into the accuracy–resource trade-off across three dimensions: output representation, quantization bit-width, and inference strategies.

1. Output Representations

The study evaluates three encoding schemes on a compact fully-convolutional backbone (128×128 output grid):

Heatmap: A per-pixel binary classification requiring an external linker to form segments.
Center-with-Displacement: Predicts a center confidence map and two endpoint offset vectors (four channels).
F-Clip (Center-Length-Angle): Encodes each segment as a center confidence, length ( $\ell$ ), and orientation. Crucially, the angle is encoded as a double-angle vector $(\cos 2\theta, \sin 2\theta)$ to resolve 180° ambiguity. This formulation requires only four output channels and avoids external linking.

2. Architecture and Quantization

Backbone: A compact, strided fully-convolutional network (5 layers) with channel widths scaling from 8 to 32. The smallest variant contains ~25k parameters.
Quantization: The model employs per-tensor symmetric integer quantization. The authors evaluate full-precision (fp32), 8-bit (int8), and 4-bit (int4) precision. They utilize Quantization-Aware Training (QAT) with straight-through estimators (STE) to simulate quantization during training.
Deployment: Training occurs off-device (GPU), while inference runs on-device (STM32F746 and STM32H7) using CMSIS-NN for optimized integer arithmetic.

3. Inference Enhancements (MiLSD System)

For the STM32H7 (1 MB SRAM budget), the system integrates:

Capacity Scaling: Increasing backbone width to ~0.39M parameters.
Sub-pixel Decoding: Refining peak locations via parabolic fitting.
Test-Time Augmentation (TTA): Averaging predictions over horizontal, vertical, and diagonal flips.
Line-of-Interest (LoI) Verification: A lightweight MLP head that pools features along candidate lines to re-score and filter spurious detections.

Key Results

Representation and Capacity

F-Clip Superiority: Under extreme memory constraints (25k parameters), the F-Clip representation significantly outperforms Heatmap and Center-with-Displacement methods. It achieves an sAP10 of 10.6, whereas the heatmap baseline scores 0.3 and the endpoint method scores 3.6. This indicates that explicit geometric encoding is more critical than model capacity at this scale.
Capacity Limits: Increasing parameters from 25k to ~209k yields negligible gains in held-out loss, suggesting the model hits an information-theoretic limit imposed by the dataset resolution and input size rather than a capacity bottleneck.

Quantization Sensitivity

8-bit Performance: int8 quantization preserves full-precision performance with negligible degradation (sAP10 drops from 10.6 to 10.0).
4-bit Failure: 4-bit quantization causes catastrophic failure (sAP10 drops to 0.7 with Post-Training Quantization). Even with QAT, performance only recovers to 6.9.
Angle Regression: The degradation is concentrated in the $(\cos 2\theta, \sin 2\theta)$ angle regression branch, which is highly sensitive to the coarse 16-level grid of 4-bit quantization.

System Performance (MiLSD on STM32H7)

By scaling the backbone to 1 MB activation budget and applying inference enhancements, MiLSD achieves:

sAP10 = 24.1 on the ShanghaiTech Wireframe benchmark.
This represents a significant improvement over the 0.25 MB baseline (10.6) while remaining within the 1 MB SRAM limit of the STM32H7.
The LoI verification head contributes the largest single gain (approx. 3.1 points), bridging the gap between raw detection and verified segments.

Significance and Claims

The paper positions MiLSD as the first learned line-segment detector explicitly designed and evaluated under sub-megabyte MCU SRAM budgets. Its primary contributions are:

Mapping the Trade-off: It characterizes the accuracy–resource frontier across representations, quantization, and post-processing, identifying that F-Clip is the optimal representation for small models but is fragile to 4-bit quantization in the angle channel.
Feasibility Demonstration: It proves that meaningful wireframe quality (sAP10 = 24.1) is achievable on embedded hardware, occupying a previously empty region between classical lightweight detectors and high-accuracy GPU parsers.
Modest Expectations: The authors acknowledge that while MiLSD is a breakthrough for embedded vision, its accuracy (24.1) remains substantially below GPU-class parsers (e.g., 60–70 sAP10). This gap is attributed to the inherent limitations of the memory regime (kilobytes vs. megabytes) rather than a failure of the method.

The work concludes that for MCU-scale detection, geometric encoding choices and quantization sensitivity are more critical determinants of success than raw parameter count, and that 8-bit quantization is a necessary threshold for viable geometric regression in this domain.