💬 NLP

AI Appeals Processor: A Deep Learning Approach to Automated Classification of Citizen Appeals in Government Services

This paper introduces AI Appeals Processor, a microservice-based system that leverages a Word2Vec+LSTM deep learning architecture to automatically classify and route citizen appeals with 78% accuracy, significantly reducing processing time compared to traditional manual methods and offering a more efficient alternative to transformer-based models.

Original authors: Vladimir Beskorovainyi

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Vladimir Beskorovainyi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive, bustling post office where citizens drop off millions of letters every day. These letters aren't just "Hello, how are you?" notes; they are urgent requests, angry complaints, and creative ideas for the government.

The Problem: The Overwhelmed Sorters
In the past, this post office relied entirely on human workers to read every single letter, figure out what it was about, and decide which department should handle it.

The Bottleneck: It took these workers about 20 minutes to process just one letter.
The Mistakes: Even after all that time, they only got the right department about 67% of the time.
The Result: A giant pile of unsorted mail, frustrated citizens, and a government that couldn't keep up with the demand.

The Solution: The "AI Appeals Processor"
The authors of this paper built a smart robot assistant (a software system) to help sort these letters. They didn't just build one robot; they tested five different "brains" to see which one worked best.

Think of these different approaches like different ways of teaching a child to read:

The "Keyword Counter" (BoW + SVM): Like teaching a child to sort letters by counting how many times words like "broken" or "help" appear. It's simple but misses the nuance.
The "Word Importance" Sorter (TF-IDF + SVM): A slightly smarter version that knows "broken" is more important than "the."
The "Fast Learner" (fastText): This robot looks at the tiny building blocks of words (like letters and syllables) to guess meaning, even if the spelling is weird. It learns incredibly fast.
The "Context Reader" (Word2Vec + LSTM): This is the winner of the competition. Imagine a librarian who doesn't just look at individual words but understands the story and flow of the sentence. It knows that "The elevator is broken, please fix it" is a complaint, while "The elevator is broken, I am writing to report it" is a formal application.
The "Super-Genius" (BERT): This is a massive, highly educated robot that knows almost everything about language. It is the smartest, but it's also slow, expensive to feed (requires huge computer power), and takes a long time to train.

The Race Results
The researchers tested these robots on 10,000 real letters. Here is what happened:

The Human Baseline: 67% accuracy, 20 minutes per letter.
The "Super-Genius" (BERT): Got the best score (82% accuracy), but it was too heavy and slow for the job.
The Winner (Word2Vec + LSTM): It hit a sweet spot. It achieved 78% accuracy (a huge jump from humans) and cut the processing time in half to just 10 minutes.

How It Works in Real Life
The system isn't just a magic box; it's a team effort.

The Robot Reads: The AI scans the letter, cleans up the grammar, and guesses the category (Complaint, Application, or Proposal).
The Human Checks: Because the robot isn't perfect (it still gets confused by tricky letters), a human operator quickly reviews the robot's guess.
The Learning Loop: This is the secret sauce. When the human corrects the robot, the robot learns from that mistake. Over time, the robot gets smarter and needs less help. It's like a student who gets better every time a teacher corrects their homework.

Why This Matters

Speed: The system processes letters 54% faster than humans alone.
Scalability: Even if 100 people send letters at the exact same time, the system doesn't crash; it just queues them up efficiently.
Accuracy: It handles specific topics like "Housing" or "Healthcare" very well (84% accuracy), though it struggles a bit with vague "Miscellaneous" letters.

The Bottom Line
This paper proves that you don't need the most expensive, super-complex AI to solve government problems. By choosing a "Goldilocks" solution—not too simple, not too heavy—they built a system that is fast, accurate, and actually saves money. It turns a chaotic pile of letters into an organized, efficient flow, freeing up human workers to do the complex work that robots can't do yet.

1. Problem Statement

Government agencies globally are facing a surge in electronic citizen appeals (complaints, applications, and proposals), with volumes increasing up to 3.5 times in some jurisdictions. Current manual processing is inefficient and error-prone:

Bottlenecks: Average processing time is 20 minutes per appeal.
Low Accuracy: Manual classification accuracy is approximately 67%, leading to misrouting and delays.
Complexity: Russian-language appeals present unique challenges due to complex morphology (inflectional forms), domain-specific terminology, and ambiguous intent (e.g., mixing complaints with requests).
Need: There is a critical need for an automated system that improves classification accuracy and speed while integrating seamlessly with existing government information systems.

2. Methodology

The authors developed AI Appeals Processor, a microservice-based system designed for the Russian language context.

System Architecture

Structure: A microservice web application with five components: a JavaScript frontend, an object-oriented backend, a Python/TensorFlow AI module, a relational database, and a REST API integration layer.
Deployment: Containerized using Docker and orchestrated via Kubernetes to ensure horizontal scaling and high availability.
Pipeline:
1. Preprocessing: Normalization, tokenization, stop-word removal (NLTK Russian list), and lemmatization (using pymorphy2 to handle Russian morphology).
2. Feature Extraction: Conversion of text to numerical vectors via TF-IDF or Word2Vec.
3. Classification: Prediction of appeal type and thematic domain.
4. Post-processing: Storage and routing via message brokers for human operator review.

Experimental Setup

Dataset: 10,000 real, anonymized citizen appeals from a government agency.
- Labels: 3 primary types (Complaints, Applications, Proposals) and 7 thematic domains (e.g., Housing, Healthcare).
- Split: 70% Training, 15% Validation, 15% Test (stratified).
- Ground Truth: Established via a two-stage process (operator labeling + expert verification), with inter-annotator agreement ( $\kappa$ ) of 0.81.
Models Evaluated:
1. Baseline: Manual human classification.
2. Traditional ML: Bag-of-Words (BoW) + SVM, TF-IDF + SVM.
3. Deep Learning:
  - fastText: Character n-grams (3–6) on raw text.
  - Word2Vec + LSTM: 128-dim embeddings fed into a unidirectional LSTM (64 hidden units) with dropout.
  - BERT: Fine-tuned bert-base-multilingual-cased.

3. Key Contributions

Comprehensive Evaluation: A rigorous comparison of five distinct ML approaches on a large, real-world Russian government dataset.
Production-Ready Architecture: A fully containerized, scalable microservice system integrating NLP with government document management systems (EDMS).
Human-in-the-Loop (HITL) Framework: A design where operator corrections feed back into the training pipeline, enabling iterative model improvement post-deployment.
Domain-Specific Benchmarks: Detailed performance analysis across seven thematic domains, highlighting the impact of specialized vocabulary on model accuracy.

4. Key Results

Classification Performance

Best Balanced Model: Word2Vec + LSTM was selected for production.
- Accuracy: 78% (an 11% improvement over the 67% manual baseline).
- Efficiency: Achieved a strong balance between accuracy and computational cost.
Top Performer: BERT achieved the highest accuracy (82%) but required significantly more resources (240 mins training time vs. 95 mins for LSTM) and complex infrastructure.
Surprising Contender: fastText achieved 76% accuracy with only 3 minutes of training, outperforming TF-IDF+SVM (75%) despite lacking explicit morphological preprocessing, thanks to its character n-gram features.

Processing Time & Scalability

Time Reduction: The automated system reduced average processing time by 54% (from 22.5 min to 10.25 min per appeal).
- Note: ML inference takes <2 seconds; the remaining time is dominated by necessary human review and routing.
Scalability: Under concurrent load, the system showed graceful degradation. At 20 simultaneous requests, latency increased by only 25%. Even at 100 concurrent requests, the system remained faster than manual processing.

Domain-Specific Analysis

High Performance: Domains with specialized terminology (Housing & Utilities, Healthcare) achieved F1-scores of 84% and 80%, respectively.
Low Performance: The "Miscellaneous" category scored lowest (F1: 71%) due to a lack of clear thematic boundaries.
Error Analysis: The primary errors stemmed from "dual-intent" appeals (e.g., complaining while requesting repair) and formal bureaucratic phrasing that obscures intent.

5. Significance and Conclusion

The paper demonstrates that deep learning can effectively modernize government services in linguistically complex environments like Russia.

Practical Impact: The system successfully deployed in a government agency, proving that a Word2Vec+LSTM architecture offers the optimal trade-off for resource-constrained environments compared to heavy transformer models like BERT.
Continuous Improvement: The "Human-in-the-Loop" mechanism proved vital; iterative retraining on operator-verified data further improved accuracy beyond the initial research metrics.
Future Directions: The authors suggest expanding training data, implementing finer-grained subcategory classification, and adopting Russian-specific pre-trained models (e.g., RuBERT) to handle ambiguity better.

In summary, the AI Appeals Processor provides a robust, scalable, and accurate solution that significantly reduces administrative burdens while maintaining the necessary human oversight for public service delivery.