💬 NLP

Enabling Stroke-Level Structural Analysis of Hieroglyphic Scripts without Language-Specific Priors

This paper introduces HieroSA, a generalizable framework that enables Multimodal Large Language Models to automatically extract interpretable, stroke-level structural representations from hieroglyphic and logographic character images without relying on language-specific priors or handcrafted data.

Original authors: Fuwen Luo, Zihao Wan, Ziyue Wang, Yaluo Liu, Pau Tong Lin Xu, Xuanjia Qiao, Xiaolong Wang, Peng Li, Yang Liu

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Fuwen Luo, Zihao Wan, Ziyue Wang, Yaluo Liu, Pau Tong Lin Xu, Xuanjia Qiao, Xiaolong Wang, Peng Li, Yang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are looking at an ancient Egyptian hieroglyph or a complex Chinese character. To a human, it's a picture with meaning. To a standard AI (like the ones that power chatbots today), it's just a grid of colored pixels or a random code symbol. The AI sees the "skin" of the character, but it is completely blind to the "bones"—the individual lines and strokes that make it up.

This paper introduces a new tool called HieroSA (Hieroglyphic Stroke Analyzer) that teaches AI to see these characters not as blurry pictures, but as lego structures made of distinct sticks.

Here is a simple breakdown of how it works, why it matters, and what it can do.

1. The Problem: AI is "Stroke-Blind"

Think of a modern Large Language Model (LLM) like a person who has memorized a dictionary but has never learned how to draw. If you show them a drawing of a house, they know the word "house," but they don't understand that a house is built from a roof, walls, and a door.

Current AI: Sees a character as a blob of pixels. It can guess the word, but it doesn't understand how the word is constructed.
Old Methods: Tried to teach AI the rules of writing (e.g., "Chinese characters always have a vertical line first"). But this only works for languages we already know well. If you show it an ancient, forgotten script, the AI gets confused because it doesn't know the rules.

2. The Solution: The "Digital Skeleton"

The researchers built a system that acts like a digital X-ray. Instead of asking the AI to guess the rules, they let it learn by trial and error, using a technique called Reinforcement Learning (think of it like training a dog with treats).

The Goal: The AI is given a black-and-white image of a character. Its job is to draw a set of straight lines (strokes) that perfectly cover the black parts of the image.
The Reward System:
- If the AI draws a line that covers black ink? Treat! (Good job).
- If the AI draws a line that goes into the white background? No treat. (Bad job).
- If the AI draws too many unnecessary lines? Penalty. (Stop wasting time).
The Result: The AI learns to break the character down into its simplest building blocks (line segments) without anyone ever telling it what a "stroke" is. It figures out the structure on its own.

3. The Magic: One Tool for All Languages

The coolest part is that HieroSA doesn't need a manual.

Imagine you have a puzzle box. Usually, you need a specific instruction manual for the "Chinese" box and a different one for the "Egyptian" box. HieroSA is like a universal puzzle solver. Because it learns purely from the visual shapes (the black ink), it can take a modern Chinese character, a Japanese Kanji, or a 3,000-year-old Oracle Bone Script, and break them all down into lines using the exact same logic. It doesn't need to speak the language; it just needs to see the picture.

4. What Can We Do With This?

Once the AI understands the "bones" of the characters, it can do some really cool things:

Better Reading (OCR): Just like a human reads faster when they understand the structure of a word, the AI reads ancient or blurry text much better when it understands the strokes. It's like giving the AI a pair of glasses that sharpen the edges of the letters.
Finding Hidden Cousins: This is the most exciting part. HieroSA can find characters that look different but share the same "skeleton."
- Example: It might find two ancient Egyptian symbols that look totally different to us. But when HieroSA breaks them down, it sees they both use the same "foot" shape in the middle. This tells linguists that these two words are related, even if we didn't know that before. It's like finding a family resemblance between two people who look nothing alike at first glance.

The Big Picture

In short, HieroSA is a translator that speaks the language of structure instead of the language of words.

It allows computers to look at ancient, forgotten, or complex writing systems and say, "I don't know what this word means, but I know exactly how it's built." This opens the door for AI to help us decipher lost histories and understand the deep logic of how humans have used pictures to communicate for thousands of years.

1. Problem Statement

Hieroglyphic and logographic writing systems (e.g., Chinese characters, Japanese Kanji, Oracle Bone Script, Egyptian Hieroglyphs) encode rich semantic information within their internal structural composition. However, current state-of-the-art Large Language Models (LLMs) and Multimodal LLMs (MLLMs) fail to effectively model this structure:

LLMs treat characters as abstract textual tokens, discarding internal structural information.
MLLMs process characters as raw pixel grids, capturing visual appearance but failing to model the underlying logic of stroke relationships or compositional principles.
Existing Structural Analysis Methods are often script-specific, relying heavily on labor-intensive manual annotations, predefined stroke inventories, or linguistic priors. This limits their scalability and generalization to ancient or less-studied scripts where standard stroke definitions are ambiguous or non-existent.

The core challenge is to develop a generalizable, annotation-free framework that can automatically derive stroke-level structures from character bitmaps without relying on language-specific rules or human expertise.

2. Methodology: HieroSA

The authors propose HieroSA (Hieroglyphic Stroke Analyzer), a novel framework that enables MLLMs to transform character images into explicit, interpretable line-segment representations using Reinforcement Learning (RL).

A. Stroke Representation

Geometric Abstraction: Characters are modeled as a set of line segments in a normalized coordinate space. A stroke $S$ is defined by the coordinates of its two endpoints: $S = \{(p_s^k, p_e^k)\}_{k=1}^n$ .
Flexibility: Unlike traditional methods that require predefined stroke types, HieroSA approximates complex or curved shapes by decomposing them into multiple linear segments. This allows the model to adapt to scripts with ambiguous stroke definitions (e.g., Oracle Bone Script).

B. Reward Design (Supervision without Annotations)

The training process relies solely on bitmap images (black foreground on white background) as supervision signals. The reward function is designed to maximize the overlap between generated strokes and the black pixels while minimizing invalid strokes. It consists of three steps:

Valid Stroke Identification: The model samples points along a predicted line segment. If any point falls outside the black-pixel region, the stroke is marked as invalid.
Coverage Estimation: For valid strokes, the model estimates the spatial coverage by extending the line segment tangentially and normally within the black-pixel region. Excessive extensions that intrude into neighboring strokes are truncated to form a polygonal coverage region.
Reward Aggregation:
- Coverage Reward: Measures the percentage of the black-pixel region covered by the union of valid stroke polygons.
- Penalty: Penalizes the generation of invalid strokes and redundant strokes (those that do not contribute new coverage).
- Formula: $r_s = \frac{|C_{final} \cap \Omega_B|}{|\Omega_B|} \cdot (1 - \alpha N_{invalid})$ , where $\Omega_B$ is the black pixel set and $\alpha$ is a penalty coefficient.

C. Training Paradigm

Algorithm: The authors use Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm known for improving reasoning in LLMs.
Base Model: Qwen3-VL-4B-Instruct is used as the foundation.
Objective: The model is trained to output structured line-segment data (e.g., JSON-like lists of coordinates) that maximize the reward $r = r_s + \beta r_f$ , where $r_f$ is a format reward ensuring parseable output.

3. Key Contributions

Novel Training Framework: Introduced a method enabling MLLMs to capture stroke-level structures directly from bitmaps without manual annotations or language-specific priors.
Cross-Lingual Generalization: Demonstrated that the method generalizes effectively across diverse writing systems (Modern Chinese, Japanese Kanji, and ancient Oracle Bone Script) without re-engineering for specific scripts.
Downstream Utility: Showed that the learned stroke representations improve downstream tasks, specifically Optical Character Recognition (OCR) and structure-guided character exploration (finding semantically or structurally related characters across different scripts).

4. Experimental Results

The authors evaluated HieroSA on Chinese (ZH), Japanese (JA), and Oracle Bone Script (OBS) datasets, comparing it against strong baselines (GPT-5, Claude Sonnet 4, Qwen3-VL-4B) and computer vision line-segmentation tools (ELSED, DeepLSD).

Quantitative Performance:
- Reward (RE) & Coverage (CO): HieroSA significantly outperformed all baselines. For example, on Chinese characters, HieroSA achieved a Reward of 0.837 and Coverage of 78.5%, compared to ~0.13 and ~3-11% for GPT-5/Claude.
- Invalid Strokes (IS): HieroSA drastically reduced invalid stroke rates (e.g., 6.1% for Chinese vs. 88.2% for GPT-5).
- Geometric Metrics: On the "Make Me A Hanzi" dataset, HieroSA showed superior performance in stroke count difference (CD), mean distance (MD), and length difference (LD) compared to all other methods.
Qualitative Analysis: Visualizations confirmed that HieroSA captures the core structural skeletons and connectivity of characters, producing visually coherent decompositions even for ancient scripts where no standard stroke definitions exist.
Downstream Tasks:
- OCR: Incorporating HieroSA's stroke representations improved OCR accuracy by ~1-2% over fine-tuned baselines without stroke data.
- Character Exploration: Using stroke-derived masks for similarity search successfully identified structurally related characters across different scripts (e.g., linking Oracle Bone Script characters with similar foot-related components), revealing latent semantic relationships.

5. Significance and Impact

Democratizing Hieroglyph Analysis: By removing the dependency on expert linguistic knowledge and manual annotations, HieroSA makes structural analysis accessible for ancient, obscure, or undocumented writing systems.
Bridging Vision and Semantics: The framework provides a mechanism for MLLMs to "see" the logical construction of characters rather than just their pixel appearance, potentially unlocking deeper semantic understanding of logographic languages.
Scalability: The approach offers a scalable path for computational graphematics, enabling the systematic study of writing system evolution and the decipherment of undeciphered scripts through structural pattern matching.

In conclusion, HieroSA represents a significant step forward in computational linguistics and computer vision, proving that reinforcement learning can successfully teach multimodal models to understand the intrinsic structural logic of complex writing systems without human supervision.