💻 computer science

HACI-Net: A Hybrid Attention and Channel Interaction Network for Food Image Recognition

This paper proposes HACI-Net, a ConvNeXt-based hybrid network integrating spatial, coordinate, and channel attention mechanisms with channel interaction modules to effectively address challenges in food image recognition, achieving state-of-the-art accuracy on the VireoFood-172 and ChineseFoodNet datasets.

Original authors: Zhiyong Xiao, Chaoliang Liu, Zhaohong Deng

Published 2026-09-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhiyong Xiao, Chaoliang Liu, Zhaohong Deng

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Every day, billions of people make choices about what to eat, choices that ripple through their health, energy, and long-term well-being. For decades, tracking these choices has relied on human memory and manual logging, a process prone to error and fatigue. In recent years, scientists have turned to computers to solve this problem, teaching machines to look at a photograph of a meal and identify exactly what is on the plate. This field, known as food image recognition, aims to automate dietary monitoring, helping people manage weight, prevent chronic diseases, and understand their nutritional intake without the burden of constant manual entry. However, teaching a computer to distinguish between two very similar dishes is notoriously difficult. A bowl of noodles might look different depending on the lighting, the angle of the camera, or the clutter of a spoon and napkin in the background. Furthermore, the visual clues that humans use to tell foods apart—such as the specific texture of a sauce or the arrangement of ingredients—are often scattered across different layers of visual data, making it hard for standard computer programs to piece them together into a clear picture.

To address these persistent challenges, researchers at Jiangnan University have developed a new system called HACI-Net. This system is designed to look at food images the way a careful observer might: by focusing intently on the most important parts of the dish while ignoring the distracting background, and by carefully combining different visual clues to form a complete understanding. The team built their system upon a modern foundation known as ConvNeXt, which is already quite good at recognizing patterns in images. However, they found that this foundation alone was not enough to handle the subtle differences between similar food categories. To fix this, they added two specific tools to the system. The first is a hybrid attention mechanism, which acts like a spotlight. It scans the image to find the most significant areas, such as the main portion of food, and dims out the noise of the background, like plates or tablecloths. This ensures the computer is looking at the food itself, not the environment surrounding it.

The second tool is a channel interaction module, which helps the system connect different types of visual information. When a computer analyzes an image, it breaks the picture down into many separate channels, each capturing a specific aspect like color, shape, or texture. In older systems, these channels often worked in isolation, failing to share what they learned with one another. The new module forces these channels to talk to each other, allowing the system to realize that a specific red color and a specific smooth texture belong to the same ingredient. By mixing these signals together, the system creates a much richer and more accurate description of the food. The researchers tested this new approach on two large collections of food photographs: one containing 172 different types of dishes from various dining environments, and another containing 208 common Chinese dishes that are often visually very similar to one another.

The results showed that this combined approach works significantly better than previous methods. On the first collection of images, the new system correctly identified the food 94.17% of the time. On the more difficult second collection, where dishes look nearly identical, it achieved an accuracy of 85.46%. These numbers represent an improvement over other leading computer models, which struggled to reach these levels of precision. The researchers verified this success by creating visual heatmaps, which show exactly where the computer was looking when it made its decision. These maps revealed that the new system focused sharply on the food items, whereas older models often got distracted by background objects. While the system is currently quite large and requires powerful computers to run, the researchers acknowledge that future work will focus on making it smaller and faster so it can eventually run on everyday devices like smartphones. For now, the study demonstrates that by teaching computers to pay better attention and to share information more effectively, we can build tools that understand our diets with a level of detail previously thought difficult to achieve.

Technical Summary: HACI-Net for Food Image Recognition

1. Problem Statement

Food image recognition is a critical component of automated dietary management, nutritional assessment, and health monitoring. However, the task remains highly challenging due to several factors:

Visual Diversity and Similarity: Food categories exhibit vast intra-class variations (e.g., plating, shape, color) while often sharing subtle inter-class similarities, making fine-grained distinction difficult.
Environmental Complexity: Real-world images are captured under diverse lighting conditions, viewpoints, and backgrounds, leading to strong background interference that obscures key food regions.
Feature Limitations: Existing Convolutional Neural Networks (CNNs) often struggle with long-range dependencies, while Vision Transformers (ViTs) may suffer from feature dispersion and insufficient attention to local fine-grained details. Furthermore, both architectures often lack effective mechanisms for channel interaction, resulting in redundant representations and suboptimal feature fusion.

2. Methodology

The paper proposes HACI-Net (Hybrid Attention and Channel Interaction Network), a deep learning architecture built upon the ConvNeXt backbone. The model is designed to synergize the local feature extraction capabilities of convolution with advanced attention and channel interaction mechanisms.

2.1 Overall Architecture

HACI-Net consists of four hierarchical feature extraction stages. Each stage includes:

A downsampling layer.
Several ConvNeXt basic blocks for core feature extraction.
A Hybrid Attention Module (HAM) embedded at the end of each stage to enhance feature saliency.
Following the four stages, a Channel Interaction Block (CIB) is applied before the final classifier to optimize global feature fusion.

2.2 Hybrid Attention Module (HAM)

The HAM is designed to suppress background noise and focus on salient food regions by integrating three complementary attention mechanisms with residual connections:

Spatial Attention: Uses average pooling along the channel dimension followed by a $7 \times 7$ convolution to capture global contextual information and identify informative spatial locations.
Coordinate Attention: Decomposes spatial features into horizontal and vertical directions using global average pooling. It employs a Multi-Layer Perceptron (MLP) to model directional dependencies, preserving precise positional information.
Channel Attention: Compresses spatial dimensions via global average pooling and uses an MLP to learn inter-channel correlations, re-weighting channels based on their importance.
Residual Integration: The outputs of these branches are combined via element-wise multiplication and added back to the original features via residual connections to stabilize training and prevent feature distribution shifts.

2.3 Channel Interaction Module (CIB)

To address the limitation of ConvNeXt in modeling inter-channel dependencies, the CIB facilitates dynamic information exchange across channels:

Pointwise Grouped Convolution: Expands the channel dimension to enable preliminary cross-channel interaction while reducing computational complexity.
Channel Shuffle: Rearranges channels across groups to alleviate channel isolation and promote comprehensive information exchange.
Depthwise Convolution: Captures local spatial context independently for each channel, preserving channel independence and reducing parameters.
MLP and Normalization: A second $1 \times 1$ grouped convolution compresses the features, followed by Layer Normalization, a residual connection, and an MLP to model complex non-linear inter-channel dependencies.

3. Key Contributions

The authors summarize their contributions as follows:

Novel Architecture: Proposal of HACI-Net, a ConvNeXt-based network specifically tailored for food image recognition.
Hybrid Attention Design: Development of a module integrating spatial, coordinate, and channel attention with residual connections to enhance focus on key regions and improve training stability.
Channel Interaction Mechanism: Introduction of a module utilizing pointwise grouped convolutions and channel shuffling to enable effective cross-channel information exchange.
Empirical Validation: Demonstration of competitive accuracy and strong generalization on two major datasets.

4. Experimental Results

The model was evaluated on two public datasets: VireoFood-172 (172 categories, real-world dining environments) and ChineseFoodNet (208 Chinese dishes, high visual similarity).

Performance Metrics: HACI-Net achieved recognition accuracies of 94.17% on VireoFood-172 and 85.46% on ChineseFoodNet.
Comparison: The model outperformed state-of-the-art baselines, including ResNet, TResNet, EfficientNet, ViT, Swin Transformer, and DeiT variants, as well as previous food-specific methods like MSMVFA and MVANet.
Ablation Studies: Experiments confirmed that both the Hybrid Attention and Channel Interaction modules contribute positively to performance. The combination of both modules yielded the highest accuracy, validating their synergistic effect.
Visualization: Heatmap analysis (Grad-CAM) indicated that HACI-Net focuses more accurately on main food components and is less distracted by background elements compared to the baseline ConvNeXt.

5. Significance and Future Work

The paper claims that HACI-Net effectively addresses the challenges of background interference, class similarity, and weak semantic correlations in food image recognition. By integrating hybrid attention and channel interaction, the model achieves superior discriminative power in complex scenarios.

Limitations and Future Directions:
The authors acknowledge that HACI-Net has relatively high model complexity, posing challenges for deployment on resource-constrained mobile or edge devices. Future work is proposed to focus on:

Lightweight Design: Exploring neural architecture search, knowledge distillation, channel pruning, and low-bit quantization to reduce computational costs.
Multimodal Integration: Moving beyond single-modality visual input to incorporate auxiliary information such as ingredient composition, nutritional labels, and recipe texts using image-text alignment and cross-modal attention. This aims to expand the system from simple recognition to comprehensive nutritional understanding and personalized dietary recommendations.