Hybrid E-Assessment in Higher Education: Semi-Automated Grading of Paper-Based Written Examinations

Technical Summary: Hybrid E-Assessment in Higher Education

Problem Statement
The paper identifies a critical tension in the transition to summative e-assessment in higher education: the conflict between the need for automated efficiency and the didactic necessity of complex, open-ended problem-solving tasks. Fully digital approaches, while offering high automation, are often constrained to closed question formats (e.g., multiple-choice) due to the inability of current systems to reliably grade free-text, mathematical derivations, sketches, or complex reasoning processes. This limitation forces a "didactic narrowing," where assessment tasks are redesigned to fit technical constraints rather than pedagogical goals. Furthermore, fully digital summative exams impose significant organizational and technical burdens, including the requirement for large numbers of synchronized workstations, strict invigilation protocols, and robust security against manipulation, which are particularly challenging for large student cohorts. Conversely, partially digital approaches (e.g., scan-based systems) often suffer from rigid page layouts and low question density, failing to replicate the flexibility of traditional paper examinations.

Methodology
The authors propose a hybrid e-assessment approach that retains the didactic flexibility of paper-based examinations while introducing semi-automated grading. The methodology operates on the following principles:

1. Structured Gap Coding: Examination tasks are designed as open, problem-oriented problems (e.g., derivations in physics or database queries). However, specific assessment-relevant intermediate results are encoded into a structured format. Students perform the full reasoning and calculation by hand but must transfer their final answers for specific steps into predefined answer fields.
2. Letter-Based Answer Options: Instead of free-text entry, each gap corresponds to a fixed set of letter-based options (e.g., A–Z). This large option pool minimizes the probability of correct answers via random guessing without forcing the entire task into a closed format.
3. Data Capture and Recognition: The handwritten letters entered by students into the answer table are captured via scanning. The core technical challenge addressed is the reliable recognition of these handwritten characters under realistic examination conditions.
4. Validation Mechanism: The system employs a two-pass validation principle combined with vision-capable large language models (LLMs). The recognition process compares the captured characters against a solution key to reduce misclassifications.

Key Contributions

• Hybrid Architecture: The paper introduces a workflow that decouples the design of the task (which remains open and paper-based) from the capture of the answer (which is structured for automation). This allows for the assessment of complex reasoning processes without sacrificing the efficiency of automated grading.
• Integration of Vision-Capable LLMs: The authors highlight the application of recent vision-capable large language models to solve the bottleneck of handwritten character recognition in a high-stakes testing environment, moving beyond traditional Optical Character Recognition (OCR) limitations.
• Didactic Preservation: By utilizing a "coding format" for intermediate results, the approach preserves the learning-oriented nature of the examination (where students must derive the solution) while enabling the automation of the grading workflow.

Results and Claims
The paper does not present empirical data from a specific large-scale deployment but rather offers a theoretical and technical analysis of the proposed framework. The authors claim that this approach:

• Reduces Misclassifications: The combination of vision-capable LLMs, a two-pass validation principle, and comparison against a solution key significantly improves the reliability of recognizing handwritten characters.
• Improves Validity and Fairness: By allowing complex, problem-oriented tasks to be graded semi-automatically, the system supports a more valid measurement of student competence compared to the restricted formats of fully digital exams.
• Enhances Scalability: The method addresses the organizational constraints of large cohorts by automating the grading of structured intermediate results without requiring a fully digital examination environment.

Significance
The significance of this work lies in its proposal to reconcile the opposing demands of modern assessment: the need for scalable, automated grading and the requirement for didactically rich, open-ended tasks. The paper argues that technology should support the grading and post-processing of examinations without constraining the design of the tasks themselves. By maintaining the paper format for the problem-solving process and only digitizing the structured capture of results, this hybrid model offers a viable pathway for implementing semi-automated grading in summative assessments where fully digital solutions are currently insufficient.