💻 computer science

Multi-Modal Semantic Segmentation of Electrolyzer Components for Sustainable Hydrogen Technologies: A Dual-Branch Deep Learning Approach

This paper proposes HREM-Net, a dual-branch deep learning framework that fuses hyperspectral and RGB imaging with advanced attention and fusion modules to achieve high-accuracy segmentation of electrolyzer materials, thereby enabling automated disassembly and sustainable recycling for hydrogen technologies.

Original authors: Wasimul Karim, Nur Mohammad Fahad, Abdul Hasib Siddique, Md Rafiqul Islam, Hooman Mehdizadeh-Rad, Asif Karim, Sami Azam

Published 2026-07-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wasimul Karim, Nur Mohammad Fahad, Abdul Hasib Siddique, Md Rafiqul Islam, Hooman Mehdizadeh-Rad, Asif Karim, Sami Azam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to sort a giant pile of mixed-up toys, but here's the catch: some of the toys look exactly the same to your eyes. A red plastic block and a red metal block might look identical in a photo, but they are made of totally different stuff. If you want to recycle them properly, you need to know which is plastic and which is metal. This is the kind of puzzle scientists face when trying to recycle old hydrogen machines, called electrolyzers. These machines are full of different materials like steel, mesh, and special ceramics that are often crushed together. To fix them or recycle them, robots need to know exactly what each piece is. But standard cameras (the kind on your phone) can only see color and shape, which isn't enough when two materials look like twins. To solve this, scientists use a special kind of "super-vision" called hyperspectral imaging. Think of this like giving the robot a pair of glasses that can see the invisible "fingerprint" of light bouncing off every material. While a normal camera sees a red square, this super-vision sees the unique chemical recipe of that red square. By combining the sharp, clear picture from a normal camera with the secret chemical fingerprints from the super-vision, robots can finally tell the difference between materials that look identical.

This is exactly the challenge tackled in a new study by a team of researchers who built a smart computer brain to help sort these hydrogen machine parts. They created a new system they call HREM-Net. You can think of HREM-Net as a detective with two different sets of eyes working together. One eye looks at the standard red-green-blue (RGB) photo to see the shape, edges, and texture of the parts. The other eye looks at the hyperspectral data to see the hidden chemical makeup. The problem is that these two types of information are very different, and just gluing them together doesn't work well. The old way of doing this was like trying to mix oil and water; the computer got confused, especially when materials looked very similar, like black steel versus grey steel.

The researchers found that their new "two-brain" approach was much better at solving the puzzle. They designed a special system where one part of the computer focuses on the chemical fingerprints and another part focuses on the shapes. Then, they added a clever "gatekeeper" that decides how much to trust each eye at any given moment. If the lighting is weird, the gatekeeper might listen more to the chemical eye. If the shape is very clear, it might listen more to the shape eye. They also taught the computer to pay extra attention to the tiny, rare pieces that usually get ignored, using a special scoring method to make sure no material gets left behind.

When they tested this new system on a dataset of crushed electrolyzer parts, the results were impressive. The computer correctly identified the material for almost every single pixel in the image, achieving a 98.62% accuracy rate. More importantly, it got the boundaries right, meaning it knew exactly where one material ended and another began. In the tricky test of distinguishing between different types of steel, the new system reached a 91.66% accuracy, which is a huge jump compared to older methods that struggled to tell them apart at all. The researchers also tested their system on a completely different dataset involving circuit boards, and it still performed incredibly well, proving that this "two-eyed" detective doesn't just memorize one specific puzzle but can actually learn how to solve new ones.

The study suggests that this technology could be a game-changer for the future of hydrogen energy. By making it possible for robots to automatically and accurately sort these complex machine parts, we can recycle them faster and cheaper, reducing waste and helping to build a cleaner, more sustainable world. While the system works great in the lab, the authors note that real-world factories might have dust, rust, or weird lighting that could confuse the system, so there is still work to be done to make it tough enough for the factory floor. But for now, this new dual-branch deep learning approach shows that when you combine the best of two different ways of seeing the world, you can solve problems that were previously impossible.

Technical Summary: Multi-Modal Semantic Segmentation of Electrolyzer Components

Problem Statement
The accurate segmentation of electrolyzer materials is critical for the automated disassembly, sustainable recycling, and circular manufacturing of hydrogen technologies. However, this task faces significant challenges due to the strong visual similarity between different materials (e.g., Steel-Black vs. Steel-Grey), spectral overlap, irregular component shapes, and severe class imbalance within datasets. Existing approaches often rely on single-modality data (either RGB or Hyperspectral Imaging [HSI] alone), which limits their ability to distinguish visually similar materials or precisely localize boundaries. RGB images provide spatial detail but fail to distinguish materials with similar textures but different compositions, while HSI provides material-specific spectral signatures but often lacks the spatial boundary precision required for robotic sorting. Furthermore, many existing architectures lack specialized attention mechanisms and loss functions to handle the specific class imbalance and boundary errors inherent in industrial recycling datasets.

Methodology: HREM-Net
To address these limitations, the authors propose HREM-Net (Hyperspectral-RGB Electrolyzer Materials Network), a dual-branch deep learning framework designed for multi-modal semantic segmentation. The architecture processes HSI and RGB data in parallel before adaptively fusing them.

Dual-Branch Encoder:
- Hyperspectral Encoder: Designed to compress high-dimensional spectral data while preserving material-discriminative features. It employs a spectral compression stage using $1 \times 1$ convolutions followed by Efficient Channel Attention (ECA) to recalibrate channel responses. Subsequent stages utilize Mobile Inverted Bottleneck (MBConv) blocks with Squeeze-and-Excitation (SE) for spatial feature learning and Coordinate Attention to preserve positional information. Finally, Atrous Spatial Pyramid Pooling (ASPP) is applied to capture multi-scale context.
- RGB Encoder: Focuses on extracting spatial details, textures, and geometric structures. It utilizes a similar hierarchy of MBConv blocks and Coordinate Attention to maintain accurate boundary definitions, which is crucial for thin structures like meshes and electrodes.
Cross-Modal Fusion:
Instead of simple concatenation, the framework employs an attention-guided dynamic gated fusion strategy. Features from both branches are projected into a shared embedding space and processed through Coordinate Attention to enable cross-modal interaction. An adaptive gating mechanism dynamically weights the contribution of the HSI and RGB branches based on global contextual information, allowing the model to emphasize the more informative modality for specific input conditions.
Decoder and Loss Function:
The decoder reconstructs full-resolution segmentation logits using a multi-stage upsampling process with Dual Attention (DA) refinement at each stage to enhance boundary clarity. To address severe class imbalance and boundary errors, the model is trained using a composite loss function comprising:
1. PolyLoss: Augments cross-entropy with a polynomial penalty to emphasize hard examples.
2. Tversky Loss: Generalizes the Dice coefficient with asymmetric penalties to improve recall for minority classes.
3. Auxiliary Deep Supervision: Provides intermediate supervision signals to improve gradient flow and convergence.

Key Contributions

Novel Architecture: Introduction of HREM-Net, the first dual-branch HSI-RGB fusion architecture that jointly employs spectral compression and dynamic cross-modal gating for electrolyzer material segmentation.
Advanced Attention Mechanisms: Proposing a fusion method integrating ECA, Coordinate Attention, and cross-modal fusion with dynamic gating to refine feature interactions.
Feature Extraction Strategy: Utilizing MBConv blocks with SE and ASPP to capture both spatial details and multi-scale context effectively.
Optimization Strategy: Employing a composite loss function (PolyLoss + Tversky + Auxiliary Deep Supervision) to mitigate class imbalance and enhance boundary segmentation accuracy.
Performance Validation: Demonstrating state-of-the-art (SOTA) performance with consistent improvements over existing methods across different material classes, including those with highly imbalanced data.

Experimental Results
The model was evaluated on the Electrolyzers-HSI dataset (55 co-registered scenes with 5 material classes) and validated for generalization on the PCB-Vision dataset.

Electrolyzers-HSI Dataset: HREM-Net achieved a mean class accuracy of 91.66% and a mean Intersection over Union (mIoU) of 0.8211, with a pixel accuracy of 98.62%. This significantly outperformed baseline models such as U-Net, U-Net++, DeepLabV3+, and TransUNet, which reported mIoU scores ranging from 0.34 to 0.43.
Cross-Dataset Validation: On the PCB-Vision dataset, the model demonstrated strong generalization, achieving 96.91% mean class accuracy and an mIoU of 0.9396.
Ablation Studies: Experiments confirmed that the integration of all components (MBConv, ECA, ASPP, and Tversky loss) is necessary for optimal performance; removing any single component resulted in a notable drop in mIoU.

Significance and Claims
The paper posits that HREM-Net offers a viable industrial application for improving electrolyzer efficiency and predictive maintenance in hydrogen production. By enabling accurate, automated identification of visually similar materials, the framework supports robotic sorting and high-purity recovery in recycling processes, thereby reducing environmental impacts and material costs. The authors claim that their approach effectively bridges the gap between spectral material identification and spatial boundary precision, addressing limitations found in single-modality and standard ensemble approaches. While acknowledging limitations regarding data alignment and sensitivity to environmental noise (e.g., contamination, corrosion), the work establishes a robust foundation for AI-driven material segmentation in sustainable hydrogen technologies.

Technical Summary: Multi-Modal Semantic Segmentation of Electrolyzer Components

More like this