💻 computer science

ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation

The paper proposes ActiveFreq, a novel interactive segmentation framework that combines an active learning module (AcSelect) to prioritize high-impact user corrections with a frequency-domain-enhanced backbone (FreqFormer) to achieve superior accuracy and reduced human interaction in medical image analysis.

Original authors: Lijun Guo, Qian Zhou, Zidi Shi, Hua Zou, Gang Ke

Published 2026-03-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Lijun Guo, Qian Zhou, Zidi Shi, Hua Zou, Gang Ke

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to draw a perfect map of a complex city (like a medical image of a human organ) on a piece of paper. You have a very smart robot assistant that tries to draw the map for you automatically.

The Problem:
The robot is good, but not perfect. It often draws the wrong streets or misses small parks. In the past, if the robot made a mistake, a human would have to point out any mistake, and the robot would try to fix it. But here's the catch: the robot didn't know which mistake was the most important to fix. It was like a student randomly picking a wrong answer on a test to study, hoping it helps them pass. Sometimes they picked a tiny, unimportant error, and sometimes they missed the huge, game-changing mistake. This meant the human had to click the mouse many, many times to get a perfect map.

Also, the robot was only looking at the picture with "spatial eyes" (looking at shapes and colors). It was ignoring the "frequency eyes," which are like a special pair of glasses that can see the underlying rhythm and structure of the image, helping to filter out static noise and see fine details like tiny blood vessels more clearly.

The Solution: ActiveFreq
The authors of this paper created a new system called ActiveFreq. Think of it as upgrading the robot with two superpowers:

1. The "Smart Detective" (AcSelect)

Instead of letting the robot guess which mistake to fix, they gave it a Smart Detective module called AcSelect.

How it works: Imagine the robot's map has several red "error zones." The Smart Detective doesn't just pick one at random. It analyzes every single error zone and asks: "If I fix this specific mistake, how much will the whole map improve?"
The Analogy: It's like a teacher grading a test. Instead of asking the student to fix a random wrong answer, the teacher points to the specific question where the student is most confused and says, "Fix this one, and you'll understand the whole chapter."
The Result: The human only needs to click on the most "valuable" mistake. This means fewer clicks are needed to get a perfect result.

2. The "Rhythm Glasses" (FreqFormer)

The second upgrade is the robot's brain, called FreqFormer.

How it works: Usually, robots look at an image like a photograph (spatial domain). But this new robot also puts on "Rhythm Glasses" (Frequency Domain Analysis).
The Analogy: Imagine listening to a song. Looking at the image normally is like looking at the sheet music. Putting on the "Rhythm Glasses" is like listening to the audio track. You can hear the bass (the big structures) and the high notes (the tiny details) separately.
Why it helps: In medical images, there is often "static noise" (like grainy TV snow) that confuses the robot. The Rhythm Glasses filter out the high-pitched static noise while keeping the clear, low-pitched structure of the organs. This helps the robot draw the boundaries of tumors or cartilage much more precisely.

The Outcome

When the researchers tested this new system on real medical images (skin cancer scans and knee MRI scans):

Fewer Clicks: The human only had to click about 3 to 4 times to get a 90% perfect map. Previous methods needed many more clicks (sometimes double or triple that amount).
Better Accuracy: Even with just two clicks, the system was incredibly accurate, outperforming the best existing methods.
Efficiency: It's like going from a slow, winding road to a high-speed expressway. You get to the destination (a perfect medical diagnosis) faster and with less effort.

In Summary:
ActiveFreq is a smarter way for computers to learn from humans. It uses a Smart Detective to find the most important mistakes to fix, and Rhythm Glasses to see the image more clearly. This combination means doctors can get precise, life-saving diagnoses with much less time and effort.

1. Problem Statement

Interactive medical image segmentation aims to achieve precise, pixel-level labeling with minimal user input (e.g., clicks). However, existing state-of-the-art methods suffer from two critical limitations:

Inefficient Region Selection: Current approaches typically treat all mislabeled regions equally, often selecting them randomly for refinement. This fails to prioritize regions where user correction would yield the highest performance gain, leading to unnecessary user interactions.
Limited Feature Representation: Most models rely solely on spatial domain features. They overlook frequency domain information, which is crucial for filtering high-frequency noise (common in medical imaging) while preserving low-frequency structural details (e.g., blood vessels, small lesions).

2. Methodology

The authors propose ActiveFreq, a novel framework that integrates an active learning selection module with a frequency-domain-aware segmentation backbone. The pipeline consists of three main stages:

A. Coarse Segmentation (FreqFormer)

The backbone network, FreqFormer, is built upon the SegFormer architecture but introduces a FreqNet decoder.

Frequency Analysis Module (FreqModule): This module processes feature maps by splitting channels into four sub-maps.
- One sub-map retains spatial information via Depth-wise Convolution (DWConv).
- The other three sub-maps are transformed into the frequency domain using 2D Discrete Fourier Transform (DFT) along different dimension pairs: Height-Width ( $H,W$ ), Height-Channel ( $H,C$ ), and Width-Channel ( $W,C$ ).
- These frequency features are weighted, converted back to the spatial domain via Inverse DFT (IDFT), and concatenated with the spatial features.
Goal: This multi-dimensional frequency analysis allows the network to capture global structural patterns and fine-grained details simultaneously, enhancing robustness against noise.

B. Active Learning Selection (AcSelect)

Instead of randomly selecting a mislabeled region for the next user click, AcSelect autonomously identifies the most informative region based on three entropy-based metrics calculated on the coarse mask:

Maximum Pixel Entropy (MPE): Identifies the single most uncertain pixel within a region (highest entropy).
Average Pixel Entropy (APE): Measures the overall uncertainty across all pixels in the region.
Regional Group Uncertainty (RGU): Evaluates the consistency of errors within a region by comparing the dominant error pixel to the regional average probability.

Scoring: A Region Score (RS) is calculated as a weighted sum of MPE, APE, and RGU. The region with the highest score is selected for the user to click, ensuring maximum refinement per interaction.

C. Refinement Process

Once the user clicks on the selected region, a lightweight refinement layer updates the mask. It employs ROI Align to extract features from the specific region of interest without quantization errors, ensuring that previously correct regions remain unaffected while the targeted area is refined.

3. Key Contributions

ActiveFreq Framework: The first interactive segmentation pipeline that prioritizes mislabeled regions based on informativeness rather than random selection, significantly reducing user interaction costs.
AcSelect Module: A plug-and-play active learning module that utilizes a multi-level entropy analysis (pixels, pixel sets, and regions) to guide user clicks toward the most impactful areas.
FreqFormer Architecture: A novel backbone integrating Fourier Transform modules to fuse spatial and frequency domain features, addressing the limitations of spatial-only models in complex anatomical segmentation.
State-of-the-Art Performance: Demonstrated superior efficiency and accuracy on medical benchmarks compared to existing methods like FocalClick, SimpleClick, and GraCo.

4. Experimental Results

The method was evaluated on two medical datasets: ISIC-2017 (dermoscopy images) and OAI-ZIB (3D knee MRI).

Efficiency (Number of Clicks - NoC):
- ISIC-2017: Achieved 3.74 NoC@90, a 23.5% improvement over the previous best (GraCo).
- OAI-ZIB: Achieved 9.27 NoC@90, a 12.8% improvement over the previous best (FocalClick).
Accuracy with Minimal Input:
- With only 2 clicks, ActiveFreq achieved 85.29% mIoU on ISIC-2017 and 75.76% mIoU on OAI-ZIB, outperforming all competitors.
Ablation Studies:
- Removing the frequency module or the AcSelect metrics resulted in significant performance drops, confirming the necessity of both components.
- Fourier Transform was found to outperform Wavelet Transform in this context due to better global context capture.
Computational Cost: ActiveFreq maintains a lightweight architecture (9.20M parameters, 2.67G FLOPs), comparable to or lighter than many Transformer-based baselines.

5. Significance

Clinical Efficiency: By drastically reducing the number of clicks required to reach high accuracy (e.g., 90% IoU), ActiveFreq lowers the time burden on radiologists and pathologists, making interactive segmentation more viable for clinical workflows.
Robustness: The integration of frequency domain analysis provides a new paradigm for handling medical images with high noise and complex boundaries, offering a solution that is more robust than spatial-only approaches.
Generalizability: The AcSelect module is designed as a plug-and-play component, suggesting it can be integrated into various existing CNN or Transformer-based segmentation backbones to improve their interactive performance.

In conclusion, ActiveFreq represents a significant step forward in interactive medical image analysis by intelligently guiding user input and leveraging frequency domain features to achieve high-precision segmentation with minimal human effort.