Empirical Investigation of Vision Architectures for High- Cardinality Fine-Grained Ancient Chinese Ideograph Classification on OBWX

For centuries, scholars have studied the earliest written records of Chinese civilization, the intricate symbols carved onto turtle shells and bronze vessels thousands of years ago. These ancient characters are not merely pictures; they are a complex system where a single concept can appear in dozens of slightly different forms depending on the era, the region, or the material used. The challenge for modern researchers is to teach computers to recognize these symbols automatically. This is not a simple task of matching a picture to a label. The system must distinguish between thousands of visually similar characters, many of which appear only a handful of times in the available historical records, while others appear frequently. This creates a difficult imbalance where the computer sees some characters often but rarely sees others, making it hard to learn the full set. Furthermore, the quality of these ancient images varies wildly, with some damaged by time, ink smudges, or missing strokes, adding another layer of confusion for any machine trying to learn from them.

A team of researchers has now built a rigorous testing ground to see which modern computer vision systems are best suited for this specific historical puzzle. They focused on a massive collection of over 74,000 images representing 6,650 distinct ancient character categories, drawn from four major historical periods ranging from the earliest oracle bones to the later small seal script. To ensure a fair comparison, they put fourteen different computer architectures through the exact same training and testing routine, holding every variable constant so that the results would depend solely on the design of the system itself. The goal was to move beyond isolated experiments and establish a clear, reproducible ranking of which approaches actually work best for this high-stakes, fine-grained classification task.

The results revealed a clear hierarchy among the different computer designs. The system that performed best was a modern convolutional network, a type of architecture that processes images by scanning them with small filters to detect local patterns like lines and curves. This model achieved an accuracy of roughly 74.5 percent, correctly identifying the vast majority of the test characters. It was followed by a hybrid system that combines convolutional scanning with attention mechanisms, and then by a hierarchical transformer, which processes images in a structured, layered way. Interestingly, the most powerful and largest models did not win. One of the largest systems, which uses a pure attention-based approach and has four times as many adjustable parameters as the winner, performed significantly worse, falling short by more than nine percentage points. This finding suggests that for this specific task, the way a model is built matters far more than how big it is. A model with fewer parameters but a design optimized for local details can outperform a massive system that tries to process the whole image at once.

The researchers also discovered that the difficulty of the task is not spread evenly across all characters. The system performed exceptionally well on the most common characters, which appeared frequently in the training data, achieving over 90 percent accuracy on these "head" categories. However, performance dropped steadily as the characters became rarer. For the least common characters, which made up the majority of the dataset, accuracy fell to about 60 percent. These rare categories were responsible for nearly two-thirds of all the mistakes the system made. The errors were not random; they almost always happened between different characters that looked very similar, such as two symbols that differ by only a single stroke or a small component. The system rarely confused different versions of the same character, but it frequently struggled to tell apart two distinct characters that shared a similar visual structure. This indicates that the primary bottleneck is not the variation within a single character over time, but the ability to distinguish between different characters that look nearly identical.

When the researchers tested how these systems held up against degraded images, simulating the damage found on real ancient artifacts, the rankings changed. The system that was best on clean images did not remain the most robust when noise was introduced. A hybrid model that ranked second on clean data proved to be the most resilient, retaining a higher percentage of its accuracy when the images were blurred or had missing parts. In contrast, the top-performing model on clean data suffered a sharper decline when the images were damaged, particularly when strokes were eroded. This suggests that the design features that make a model excellent at recognizing perfect images do not necessarily make it the most reliable for real-world historical documents, which are often imperfect.

The study also examined whether the results were stable or just a lucky outcome of a specific training run. By repeating the entire experiment with different random starting points, the researchers found that the top performers remained consistent. The best model stayed at the top, and the group of top three models did not change, regardless of how the training was initiated. However, some of the lower-performing models showed wild swings in performance depending on the random seed, with their accuracy varying by as much as twenty-five percentage points between runs. This instability suggests that while the leading designs are robust and reliable, some of the older or less optimized architectures are prone to failure and cannot be trusted to produce consistent results without careful tuning.

Ultimately, this work provides a clear map for future research in digital paleography. It shows that for recognizing ancient Chinese characters, a model that balances efficiency with local detail detection is superior to simply using the largest available system. It highlights that the greatest challenge lies in distinguishing between visually similar, rare characters rather than in handling the variation of common ones. By establishing a standardized benchmark, the researchers have given the community a common language to discuss progress, ensuring that future improvements are measured against a solid, reproducible foundation rather than isolated, incomparable experiments. The path forward involves using these insights to develop specialized methods for the rare characters and to build systems that can withstand the physical degradation inherent in the historical record.