💻 computer science

DeepNeutroBiLSTM: A Hybrid Neutrosophic and CNN-BiLSTM Framework with Sobel-Based Indeterminacy for Arabic Handwritten Character Recognition

This paper proposes DeepNeutroBiLSTM, a novel hybrid framework integrating neutrosophic uncertainty modeling with Sobel-based indeterminacy, CNNs, and BiLSTMs to achieve state-of-the-art accuracy and robustness in Arabic handwritten character recognition on the AHCD and HIJJA datasets.

Original authors: Othmane Farhaoui, Mohamed Rida Fethi, Imad Zeroual, Ahmad El Allaoui

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Othmane Farhaoui, Mohamed Rida Fethi, Imad Zeroual, Ahmad El Allaoui

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to read handwritten Arabic letters. This is a tricky job because Arabic is written in a flowing, cursive style, and people write very differently from one another. Some letters look almost identical, and sometimes the ink bleeds or the scanner makes the image fuzzy. To a computer, a blurry or messy letter is like a puzzle with missing pieces.

This paper introduces a new computer program called DeepNeutroBiLSTM to solve this puzzle. Here is how it works, broken down into simple concepts:

1. The Problem: "Everything Looks the Same"

Most computer programs that read handwriting treat every single dot of ink on the page as equally important. They assume the image is perfect. But in reality, handwriting is messy.

The Analogy: Imagine trying to read a handwritten note where someone spilled coffee on it. A standard computer tries to read the coffee stain as if it were part of the letter. It gets confused because it doesn't know what is a real letter and what is just a messy stain.

2. The Solution: The "Three-Layer Glasses" (Neutrosophic Representation)

The authors created a special way for the computer to look at the image, inspired by a mathematical idea called Neutrosophy. Instead of just seeing "Black" (the letter) or "White" (the background), the computer puts on a pair of "Three-Layer Glasses" that splits every part of the image into three categories:

Truth: "This part is definitely part of the letter."
Falsehood: "This part is definitely background/noise."
Indeterminacy: "I'm not sure. This looks like a smudge, a blur, or a weird stroke. I need to be careful here."

The Creative Twist: In older systems, the computer had to guess what was "uncertain" using a fixed rule (like a pre-set formula). In this new system, the computer learns what is uncertain. It's like giving the computer a student who learns from experience: "Oh, I see that this specific type of blur usually happens with this letter, so I'll flag it as 'uncertain' next time."

3. The "Sobel" Detective

To help the computer find those uncertain areas, they used a tool called the Sobel Operator.

The Analogy: Think of the Sobel operator as a detective with a magnifying glass that only looks at edges. It highlights the sharp lines where a letter starts and stops. Since Arabic letters are defined by their strokes and curves, this tool helps the computer distinguish between a real letter edge and a random ink smudge. It tells the computer, "Pay extra attention to these boundaries because that's where the confusion usually happens."

4. The Teamwork: The "Architect" and the "Storyteller"

The system combines two powerful types of artificial intelligence to do the heavy lifting:

The Architect (CNN): This part looks at the image and identifies shapes, lines, and curves. It builds a mental map of what the letter looks like.
The Storyteller (BiLSTM): Arabic is written in a flow. The shape of a letter often depends on the letters before and after it. The Storyteller looks at the sequence of shapes and understands the context. It knows, "If I see this shape after that shape, it must be this specific letter, even if they look similar."

By combining the "Three-Layer Glasses" (to handle the messiness), the "Detective" (to find the edges), the "Architect" (to see the shapes), and the "Storyteller" (to understand the flow), the system becomes very smart at reading messy handwriting.

5. The Results: How Well Did It Work?

The researchers tested their new system on two famous collections of handwritten Arabic letters (called AHCD and HIJJA).

On the "Easier" Test (AHCD): The new system got 98.39% correct. This is a huge improvement over older methods, which struggled with the messy parts of the image.
On the "Harder" Test (HIJJA): This test had much more variation in handwriting styles. The new system still managed to get 92.92% correct, beating all the other models they compared it against.

The Bottom Line

The paper claims that by teaching the computer to explicitly recognize and handle "uncertainty" (the blurry, messy parts of handwriting) rather than ignoring them, it can read Arabic handwriting much more accurately. It's like teaching a reader not just to read the words, but to also understand when the ink is smudged and to use context to figure out what the word should be.

What the paper does NOT claim:

It does not claim this works for reading entire sentences or paragraphs yet (it focuses on single characters).
It does not claim this is ready for medical diagnosis or other specific real-world applications yet; it is a proof-of-concept for character recognition.
It does not claim to be faster than all other methods (in fact, it takes a bit more time to compute because it's doing more complex analysis).

Technical Summary: DeepNeutroBiLSTM for Arabic Handwritten Character Recognition

Problem Statement
Arabic Handwritten Character Recognition (AHCR) remains a significant challenge due to the cursive nature of the script, the high variability in writing styles, the presence of diacritics, and structural similarities between characters. Furthermore, recognition systems often struggle with noise introduced by scanning, ink diffusion, and image degradation. Traditional deep learning approaches, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), typically assume that all input pixels are equally reliable. This deterministic assumption fails to account for uncertainty in ambiguous regions, leading to reduced robustness when handling degraded or variable handwriting. While neutrosophic set theory offers a mathematical framework to model uncertainty through Truth, Indeterminacy, and Falsity components, existing applications in this domain rely on manually designed, fixed rules (e.g., entropy or gradient heuristics) to calculate indeterminacy. These static methods lack the flexibility to learn uncertainty representations directly from data and cannot be jointly optimized with the recognition objective.

Methodology
The paper proposes DeepNeutroBiLSTM, a hybrid deep learning framework that integrates neutrosophic uncertainty modeling with a CNN-BiLSTM architecture. The framework operates through five primary stages: Pre-processing, Neutrosophic Representation, Feature Extraction, Classification, and Post-processing.

Learnable Neutrosophic Representation (LNL): The core innovation is the Learnable Neutrosophic Layer, which transforms a normalized grayscale input image $X$ into a neutrosophic triplet $[T, I, F]$ (Truth, Indeterminacy, Falsity).
- Truth ( $T$ ): Directly derived from the input image ( $T = X$ ).
- Indeterminacy ( $I$ ): Instead of using fixed heuristics, the paper explores two approaches. The primary contribution is a Learnable Indeterminacy component computed via a trainable convolutional mapping followed by a sigmoid activation: $I = \sigma(W_I * X + b_I)$ . This allows the network to learn where uncertainty exists directly from the data.
- Falsity ( $F$ ): Calculated as $F = \text{clip}(1 - T - I, 0, 1)$ .
- Sobel-Based Indeterminacy: For comparative evaluation, the paper also implements a fixed indeterminacy representation using the Sobel operator to capture edge gradients, which are critical for distinguishing visually similar Arabic characters.
Hybrid Architecture:
- CNN Backbone: The neutrosophic representation is fed into a hierarchical CNN (three convolutional blocks with increasing channel depth: 64, 128, 256) to extract spatial stroke patterns and structural features.
- BiLSTM Sequential Modeling: The spatial feature maps are reshaped into sequences and processed by a Bidirectional Long Short-Term Memory (BiLSTM) network. This captures contextual dependencies from both forward and backward directions, aiding in the discrimination of characters with similar shapes based on their surrounding context.
- Classification: The output is passed through fully connected layers and a Softmax classifier.
Training Strategy: The model is trained end-to-end using Categorical Cross-Entropy loss and the Adam optimizer. The framework was evaluated on two public datasets: AHCD (28 characters) and HIJJA (29 characters), with data augmentation (rotation, translation, zoom, shearing) applied to improve generalization.

Key Contributions

Learnable Neutrosophic Layer (LNL): The paper introduces a mechanism to learn the indeterminacy component directly from images using trainable convolutional filters, moving away from static, manually defined uncertainty rules.
Sobel-Based Indeterminacy Integration: The study demonstrates that integrating Sobel-based gradient information into the neutrosophic representation significantly enhances the model's ability to distinguish stroke boundaries in ambiguous regions.
Unified Hybrid Framework: The work validates a unified architecture that jointly optimizes uncertainty modeling (Neutrosophic), spatial feature extraction (CNN), and sequential context learning (BiLSTM) for AHCR.
Comprehensive Ablation Study: The authors systematically compare baseline CNNs, CNN-BiLSTMs, fixed-neutrosophic models, and the proposed hybrid models to isolate the impact of uncertainty modeling and sequential learning.

Results
The framework was evaluated on the AHCD and HIJJA datasets using Accuracy, F1-score, AUC, Character Accuracy Rate (CAR), and Expected Calibration Error (ECE).

AHCD Dataset: The proposed DeepNeutroBiLSTM (Sobel) achieved the highest performance with 98.39% accuracy, an F1-score of 0.9821, and an AUC of 0.9997. This represents a +1.96% improvement over the baseline CNN and a ~50% reduction in calibration error compared to the baseline.
HIJJA Dataset: Given the higher variability and structural ambiguity of this dataset, the proposed model achieved 92.92% accuracy and an F1-score of 0.9322. This was a significant improvement of approximately 5.0% over the baseline CNN (87.96%).
Comparative Analysis: The results indicate that while BiLSTM alone offered minimal gains on the structured AHCD dataset, the combination of neutrosophic uncertainty modeling and sequential learning provided consistent improvements across both datasets. The Sobel-based indeterminacy representation outperformed both entropy-based and learnable-only indeterminacy approaches in the final hybrid configuration.
Calibration: The proposed model demonstrated well-calibrated probability distributions, with low Expected Calibration Error (ECE) values (0.0119 for AHCD and 0.0456 for HIJJA).

Significance and Claims
The paper claims that the DeepNeutroBiLSTM framework offers a superior solution for Arabic handwritten character recognition by explicitly addressing the uncertainty inherent in visual data. By shifting from manual, fixed definitions of indeterminacy to a data-driven, learnable approach, the model can adapt to specific variations in handwriting styles and noise levels. The authors assert that the integration of Sobel-based indeterminacy is particularly effective because it emphasizes stroke boundaries and local gradient changes, which are critical for differentiating visually similar Arabic characters. The study concludes that this hybrid approach effectively handles the variability of handwriting and image noise, outperforming state-of-the-art CNN and CNN-BiLSTM baselines. The authors note that future work will focus on extending this framework to Arabic handwritten word and text recognition.