Family Balancing and Hierarchical CNNs for Arabic Handwritten Symbols: A Controlled Five-Seed Evaluation

In the world of computer vision, machines have become remarkably skilled at reading what humans write. They can scan a page of text and turn it into digital code, a capability that powers everything from scanning old documents to helping people with visual impairments read their mail. However, this technology faces a specific challenge when dealing with Arabic script. Unlike English, where letters stand alone or connect in a few predictable ways, Arabic letters change their shape depending on where they sit in a word. Furthermore, Arabic writing includes both the twenty-eight letters of the alphabet and the ten digits of the number system. When a computer tries to learn both at once, it often runs into a problem of scale: there are usually far more examples of numbers available for training than there are examples of letters. Because the computer sees so many more numbers, it tends to become very good at recognizing them while remaining slightly less accurate with the letters. This imbalance creates a blind spot in the system, where the machine fails to treat the two types of symbols with equal care.

Researchers at Mississippi State University set out to fix this blind spot. They asked a simple but difficult question: if they changed the way the computer learned, could they force it to pay more attention to the letters without losing its ability to recognize numbers? They tested two main ideas. First, they tried to balance the training process so that the computer treated errors in recognizing letters as important as errors in recognizing numbers, even though there were fewer letters to learn. Second, they tried a different way of organizing the computer's brain, making it first decide if a shape was a letter or a number before trying to identify exactly which one it was. They ran these experiments with extreme care, training the computer five separate times with slightly different starting conditions to ensure the results were real and not just a lucky fluke. They also tested the computer's resilience by showing it images that had been deliberately blurred, covered in static, or had their contrast altered, simulating the kind of damage a real-world document might suffer.

The study revealed that the computer's performance on clean, perfect images was already very high, correctly identifying symbols about ninety-eight percent of the time. However, when the researchers looked closely at the difference between how well it knew numbers versus letters, a small gap remained. The computer was consistently better at numbers. When the team tried to balance the training so the computer focused equally on both groups, the performance on letters improved slightly. The best version of this balanced approach raised the accuracy for the weaker group by a tiny fraction, moving from roughly ninety-six point three percent to ninety-six point five percent. While this sounds like a small change, in the world of high-precision machine learning, even a fraction of a percent can be meaningful. However, the researchers found that this improvement was not consistent. Depending on how the computer started its learning process, the change sometimes helped and sometimes did nothing at all. This suggested that simply telling the computer to "pay more attention" to the letters was not a guaranteed fix.

The second idea, which involved teaching the computer to first categorize a symbol as a letter or a number before identifying the specific character, did not provide a clear advantage either. The computer performed just as well, or sometimes slightly worse, when using this hierarchical method compared to the standard approach. The researchers also tested a more complex strategy where the computer was encouraged to focus extra hard on whichever group was currently struggling the most during training. This approach, designed to make the system robust against difficult examples, failed to protect the computer when the images were corrupted. In fact, when the images were blurred or had their contrast shifted, the computer using this special training lost more accuracy than the standard models.

The most striking finding of the study was not about how to improve the computer's learning, but about how fragile it was to changes in the image itself. While the different training methods produced only tiny differences in performance on perfect images, they all collapsed when the images were altered. When the researchers added noise or blurred the lines, the computer's ability to recognize the symbols dropped dramatically. In some cases, the accuracy fell to near zero. This suggests that the computer's current way of learning Arabic symbols is highly sensitive to the quality of the image. The small tweaks to the training process that the researchers tested were not enough to overcome this fundamental weakness. The computer was not just failing to learn the letters; it was failing to recognize the letters when the picture was not perfect.

The researchers concluded that while balancing the training data helped the computer recognize letters a little bit better, it did not solve the core problem. The improvements were small and inconsistent, and they did not make the system more resilient to the kinds of damage real documents often face. The study highlights that simply changing how a computer learns from data is not enough to make it robust. The next step for scientists in this field is not to tweak the training rules further, but to understand why the computer is so sensitive to changes in the image. They need to figure out how to build a system that can recognize a handwritten letter even when the ink is smudged, the paper is wrinkled, or the lighting is poor. Until then, the gap between a computer's ability to read a perfect scan and its ability to read a messy real-world document remains wide, and the specific challenge of balancing Arabic letters and numbers is just one part of that larger puzzle.