Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems

This paper presents Chitrapathak-2 and Parichay, two production-scale OCR systems for India that demonstrate through extensive evaluation that fine-tuning existing models on target languages yields superior accuracy-latency trade-offs and state-of-the-art performance across diverse Indic languages and government documents compared to end-to-end multimodal training.

The Gist

Imagine you are trying to build a super-smart robot librarian for India. This librarian's job is to look at millions of different documents—government forms, old books, handwritten notes, and official IDs—and read the text out loud perfectly.

The problem? India is like a giant, chaotic library where every book is written in a different language (22+ official ones!), uses different scripts (like Devanagari, Tamil, Telugu), and has messy layouts. Plus, the robot needs to be fast, cheap, and accurate enough to handle real-world messiness like blurry scans or crooked photos.

The authors of this paper, from Krutrim AI, tried two different ways to build this robot. They call their projects Chitrapathak (the "Image Reader") and Parichay (the "Introduction").

Here is the story of how they solved the puzzle, explained simply:

The Two Strategies: "The Generalist" vs. "The Specialist"

The team tested two main approaches to teaching the robot how to read.

Strategy 1: The "Generalist" Approach (Chitrapathak-1)

The Analogy: Imagine hiring a brilliant, multilingual professor who knows everything about the world but has never specifically studied how to read messy Indian documents. You give them a camera (vision) and a huge brain (language model) and say, "Here are some pictures of books; figure out how to read them."

• How it works: They took a powerful, generic AI model and tried to teach it OCR (Optical Character Recognition) from scratch using a "LLaVA-style" method. They fed it millions of images and let it learn the connection between pictures and text.
• The Result: It worked okay, but it was slow and clumsy. Because the "professor" wasn't a specialist, it had to think very hard about every single letter. It was like asking a genius physicist to do your laundry; they can do it, but they'll take forever and might fold the socks wrong.
• The Flaw: It struggled with high-resolution images and was too slow for real-world use.

Strategy 2: The "Specialist" Approach (Chitrapathak-2)

The Analogy: Instead of hiring a general professor, they hired a professional typist who already knows how to type fast and accurately, but only speaks English. They then gave this typist a crash course in Indian languages.

• How it works: They took an existing model that was already an expert at reading documents (Nanonets-OCR) and simply "fine-tuned" it. They didn't teach it how to see; they just taught it what to look for in Indian scripts.
• The Result: This was the winner. It was 3 to 6 times faster than the Generalist approach and actually more accurate.
• Why? The "Specialist" already knew the rules of reading documents. They just needed to learn the new vocabulary (Indian languages). It's like teaching a professional driver how to drive on the left side of the road; they don't need to relearn how to steer or brake, they just need to adjust to the new rules.

The "Parichay" Project: The ID Card Reader

While Chitrapathak is a general reader for any text, the team also built Parichay for a very specific job: reading Indian government IDs (like Aadhaar cards, Driving Licenses, and PAN cards).

The Analogy:

• Chitrapathak is like a human who reads a whole book and summarizes it.
• Parichay is like a form-filling robot. You hand it a Driving License, and it doesn't just read the text; it instantly knows, "Ah, this is the 'Name' field, and this is the 'Date of Birth' field." It ignores the rest of the page and only extracts the specific data you need.

The Magic Trick:
They added a small "pre-processor" that acts like a straightening tool. If you take a photo of an ID card at a weird angle, this tool rotates the image so it's perfectly straight before the robot reads it. This simple step made the robot much more reliable.

The Result: Parichay achieved a 90% accuracy rate in extracting specific details, beating even expensive, closed-source commercial systems, and it did it much faster.

Key Takeaways for the Real World

The paper offers three big lessons for anyone building AI systems in India:

1. Don't Reinvent the Wheel: If you want to build a reading system, don't start from scratch with a generic AI. Start with a model that is already good at reading, and just teach it your specific languages. It's faster, cheaper, and more accurate.
2. Specialization Wins: If you know exactly what you are reading (like government forms), build a specialized tool for it. Don't use a "one-size-fits-all" robot. A specialized robot is faster and makes fewer mistakes.
3. Speed Matters: In the real world, accuracy isn't enough. If your system takes 10 seconds to read a document, nobody will use it. The "Specialist" approach was not only smarter but also much quicker.

The Bottom Line

The authors successfully built a production-ready OCR system for India by realizing that you don't need a genius who knows everything; you need a skilled worker who knows exactly what to do.

By taking an existing "expert reader" and giving it a quick language lesson, they created a system that is fast, accurate, and ready to handle the chaotic, beautiful diversity of Indian documents.

Technical Summary

1. Problem Statement

Building Optical Character Recognition (OCR) systems for India presents unique challenges compared to standard Western OCR tasks:

• Linguistic Diversity: India has 22+ scheduled languages with distinct scripts (e.g., Devanagari, Telugu, Tamil, Bengali), complex ligatures, and large character inventories.
• Document Heterogeneity: Real-world documents vary wildly in layout, print quality, and language mixing (code-switching).
• Deployment Constraints: Industrial applications require strict adherence to low latency, high throughput, and cost-efficiency, which often conflicts with the computational demands of large Vision-Language Models (VLMs).
• The Gap: While general-purpose VLMs (like GPT-4o or Gemini) offer strong OCR capabilities, their performance-latency trade-offs for specific Indic scripts and structured government document extraction in production environments remain under-explored.

2. Methodology

The authors propose and evaluate two distinct training strategies using the Chitrapathak series (for general multilingual OCR) and the Parichay series (for domain-specific structured extraction).

A. Strategy 1: LLaVA-Style End-to-End Training (Chitrapathak-1)

• Architecture: Combines a generic vision encoder (CLIP-336) with a strong multilingual Language Model (Krutrim-1 7B).
• Training: Follows a two-stage process:
  1. Multimodal Pretraining: Freezes the encoder and decoder, optimizing only the projection layer.
  2. Supervised Fine-Tuning (SFT): Jointly trains the projection and decoder.
• Handling Resolution: Uses an aspect-ratio-aware tiling strategy (inspired by InternLM-XComposer2) to decompose pages into global and local crops to handle high-resolution documents.
• Limitation: The reliance on CLIP's fixed resolution and dynamic tiling creates incompatibility with optimized inference stacks (like vLLM), leading to high latency and memory overhead.

B. Strategy 2: Fine-Tuning an OCR-Specialized Model (Chitrapathak-2)

• Architecture: Fine-tunes Nanonets-OCR2-3B, a model built on the Qwen2.5-VL architecture.
• Key Features:
  • Uses a native-resolution capable vision encoder with 2D-RoPE and windowed attention.
  • Eliminates the need for dynamic tiling, allowing direct processing of document images.
  • Fully compatible with vLLM for efficient batching and memory management.
• Training: Directly supervised fine-tuned on multilingual Indic OCR data (1.1M pairs), despite the base model not being pre-trained on Indic scripts.

C. Domain-Specific Approach: Parichay

• Goal: Extract structured key fields (e.g., Name, DOB, Address) from 9 specific Indian government documents (Aadhaar, PAN, Driving License, etc.).
• Method: Formulates extraction as instruction-conditioned generation (Input: Image + Schema Prompt $\rightarrow$ Output: JSON).
• Models:
  • Parichay-1: Based on Phi-3.5 Vision Instruct (4.2B). Uses dynamic cropping and LoRA/Full Fine-tuning.
  • Parichay-2: Based on Nanonets-OCR2-3B (3B). Optimized for vLLM and low latency.
• Preprocessing: Integrates a lightweight document rotation module (based on Phi-3.5 vision encoder) to normalize document orientation before extraction, significantly improving robustness.

3. Key Contributions

1. Empirical Comparison of Training Strategies: The paper formally compares "General VLM Adaptation" (Strategy 1) vs. "OCR-Specialized Fine-Tuning" (Strategy 2). It demonstrates that Strategy 2 is superior for production, offering better accuracy-latency trade-offs and data efficiency.
2. Chitrapathak-2: A state-of-the-art (SOTA) multilingual OCR system supporting 10 Indic languages + English. It achieves a 3–6x speedup over its predecessor while maintaining SOTA accuracy (e.g., 6.69 char ANLS in Telugu).
3. Parichay Series: A specialized system for Indian government documents that achieves an 89.8% Exact Match (EM) score on structured extraction, outperforming closed-source solutions (like Gemini-2.5 Flash) with significantly faster inference.
4. Actionable Guidelines: Provides a "recipe" for practitioners, highlighting that:
   • Initializing from an OCR-specialized backbone reduces adaptation costs.
   • Tokenizer efficiency is a dominant latency factor for scripts like Telugu and Malayalam.
   • Full fine-tuning is preferred over parameter-efficient methods (LoRA) for domain-constrained, high-stakes tasks.

4. Results

Multilingual OCR (Chitrapathak)

• Accuracy: Chitrapathak-2 outperforms Chitrapathak-1, base Nanonets-OCR2-3B, and open-source models (Surya) across all 9 Indic languages tested. It rivals or beats proprietary models like Gemini-2.5 Flash and GPT-4o on most scripts.
• Latency: Chitrapathak-2 is 3–6x faster than Chitrapathak-1.
  • Example: English latency dropped from 14.38s (Chitrapathak-1) to 3.10s (Chitrapathak-2).
  • It is consistently faster than GPT-4o across all tested languages.
• Robustness: While strong on printed books, performance degrades slightly on rare scripts or complex layouts (e.g., index pages with dot leaders).

Domain-Specific Extraction (Parichay)

• Accuracy: Parichay-2 (with rotation) achieves 89.8% Exact Match, surpassing Gemini-2.5 Flash (86.0%) and the base Phi-3.5 model (23.26%).
• Efficiency: Parichay-2 achieves a 4x speedup over Parichay-1 (1.03s vs 4.10s per document) while improving accuracy.
• Impact of Rotation: Adding the rotation module increased the Mean Score from 86.48% to 92.95%.

5. Significance and Conclusion

This paper provides critical insights for deploying AI in the Indian context, where linguistic diversity and infrastructure constraints are paramount.

• Shift in Paradigm: The authors argue that for production-scale Indic OCR, specialization beats generalization. Fine-tuning a model already optimized for OCR (Strategy 2) yields better results than training a general-purpose VLM from scratch or end-to-end (Strategy 1).
• Infrastructure Alignment: The study emphasizes that model architecture must align with inference infrastructure (e.g., vLLM compatibility). Models requiring dynamic tiling or non-standard decoding paths fail to meet industrial latency requirements.
• Practical Impact: The Chitrapathak and Parichay systems offer a blueprint for building scalable, cost-effective, and highly accurate document digitization pipelines for governance and enterprise sectors in India, moving beyond research benchmarks to real-world deployment.