Technical Summary: Diffusion-Based Ukrainian Handwritten Text Generation with Cross-Domain Style Transfer

Problem Statement

Handwritten text generation (HTG) conditioned on writer style has advanced significantly for Latin scripts, particularly English, but remains underexplored for low-resource and non-Latin writing systems. A critical gap exists in understanding whether existing few-shot, writer-aware diffusion models can generalize beyond the Latin domain to scripts like Ukrainian Cyrillic. This transfer is non-trivial due to differences in character inventories, stroke patterns, and the lack of large-scale, writer-labeled datasets for Ukrainian. Furthermore, it is unclear if current models capture script-specific regularities or learn a transferable representation of handwriting style.

Methodology

Dataset Construction

To address the lack of data, the authors constructed a Ukrainian handwritten word-level dataset containing 126,177 images from 308 writers, derived from the UkrHandwritten line-level corpus. The construction pipeline involved:

1. Pre-segmentation Cleanup: Application of a NAFNet restoration network to remove faint underline artifacts that could be learned as spurious style features.
2. Connected-Component Segmentation: A custom pipeline using Otsu binarization and OpenCV to merge components within 8 pixels, followed by selecting the $N-1$ widest inter-group gaps as word boundaries. This achieved a 95.7% perfect boundary match rate, significantly outperforming vertical-projection methods (71.7%).
3. Quality Filtering: A five-stage pipeline removed samples containing Latin characters, punctuation errors, extreme aspect ratios, and low-confidence OCR matches. A length-stratified similarity threshold was applied to prevent the rejection of short function words.
4. Balancing: Rare Ukrainian characters (e.g., ф, ґ, Щ, Є, Ц, ї) were oversampled by duplicating crops 2–5 times to ensure adequate representation.

Model Architecture

The study utilized DiffusionPen, a few-shot latent diffusion model, without architectural modification.

• Base: A conditional latent diffusion model operating in the 4×8×32 latent space of a frozen Stable Diffusion v1.5 VAE.
• Conditioning:
  • Text: Encoded via a CANINE character-level encoder (which natively supports Cyrillic Unicode codepoints) and projected to the U-Net dimension.
  • Style: A continuous style embedding ($s \in \mathbb{R}^{1280}$) derived from a frozen MobileNetV2 triplet-loss encoder, mean-pooled over five reference images.
  • Writer ID: A learned label embedding summed with the style embedding.
• Training: The model was trained for 200 epochs on the new dataset using standard LDM noise-prediction objectives. Classifier-free guidance (CFG) was applied to text conditioning ($p_{drop}=0.2$), while style conditioning was never dropped.

Evaluation Protocols

The authors evaluated the model in three distinct settings to test cross-domain generalization:

1. Cross-lingual Transfer: Generating Ukrainian words using style references from the IAM English handwriting database.
2. Historical Transfer: Generating text using style references from an early 20th-century Ukrainian manuscript (different ink, paper, and letterforms).
3. Few-Shot Imitation: Generating text for unseen contemporary writers from the RUKOPYS dataset.

Key Results

Quantitative Performance

• Visual Quality: The model achieved a Fréchet Inception Distance (FID) of 23.09 on 5,000 samples across 308 writers. This is comparable to DiffusionPen's performance on the English IAM dataset (~20–25), indicating visual quality on par with Latin-script state-of-the-art.
• Legibility: The Character Error Rate (CER) measured by a Cyrillic TrOCR model was 16.0% overall.
  • Out-of-Vocabulary (OOV): 15.6% (lower than In-Vocabulary at 16.5%), confirming the model generalizes to unseen sequences rather than memorizing vocabulary.
  • Rare Letters: The CER gap between rare and common letters was only 1.4 percentage points (17.2% vs. 15.8%), validating the effectiveness of targeted oversampling.
  • Length Dependency: Performance varied by word length, with the best results in the 4–6 character range (10.8% CER) and higher errors for very short words (1–3 characters, 42.7% CER), largely attributed to recognizer limitations on single-character words.

Ablation Studies

• Dataset Size vs. Purity: Larger, noisier datasets (67K samples) outperformed smaller, ultra-clean datasets (12.8K samples), confirming that diversity dominates purity for diffusion-based HTG.
• Segmentation Quality: Switching from vertical-projection to connected-component segmentation reduced validation MSE from 0.0787 to 0.0658, identified as the single largest improvement factor.
• Architecture: Increasing the number of residual blocks per U-Net level from 1 to 2 (matching Stable Diffusion v1.5 specs) further reduced MSE and improved glyph sharpness.

Cross-Domain Transfer

The model successfully transferred styles across significant domain shifts:

• English to Ukrainian: Generated Ukrainian words visibly reproduced the stroke weight, angle, and spacing of English writers from the IAM dataset.
• Historical to Modern: The model adopted the calligraphic qualities (wider strokes, formal proportions) of a 20th-century manuscript while generating modern character forms.
• Unseen Writers: The model imitated the slant and stroke weight of writers from the RUKOPYS dataset without fine-tuning, demonstrating robust few-shot generalization.

Significance and Claims

The paper claims to provide the first empirical evidence that few-shot latent diffusion models can generalize beyond the Latin-script domain to Ukrainian Cyrillic. Key contributions include:

1. A New Benchmark: The release of a 126K-sample, writer-labeled Ukrainian word-level dataset, addressing a critical resource gap.
2. Proven Transferability: Demonstrating that a model trained on Latin scripts can be adapted to Cyrillic without architectural changes while retaining meaningful style transfer capabilities across languages, historical periods, and unseen writers.
3. Methodological Insights: Establishing that dataset diversity and segmentation quality are more critical to generation quality than strict data purity, and that matching the U-Net residual block count to the base diffusion model improves results.

The authors conclude that while limitations remain regarding rare characters and the Ukrainian apostrophe, the work provides a viable foundation for extending stylized HTG to other underrepresented writing systems.