Automatic Subjective Answer Evaluation

Abstract

Automatic evaluation of subjective answers has become a vital area of research due to its potential to reduce the manual effort required in educational assessments. This paper presents an advanced system for the automatic evaluation of handwritten subjective answers, integrating Optical Character Recognition (OCR), Natural Language Processing (NLP), and semantic similarity techniques. The system employs the Google Cloud Vision API to extract textual data from handwritten answer sheets with high accuracy. Extracted responses undergo preprocessing, including spell correction, and are semantically compared with ideal answers using both BERT and fine-tuned SBERT models. To enhance grading reliability, a custom contrastive learning mechanism is implemented for SBERT fine-tuning, using student-ideal answer pairs. The evaluation is performed via a Flask-based backend, which also supports training workflows through API endpoints. Feedback and marks are generated based on semantic similarity and model confidence. This system demonstrates an effective solution for automating subjective answer assessment with a high degree of flexibility and accuracy, particularly for handwritten inputs. The system supports both real-time evaluation and model customization, offering educators the flexibility to retrain models using domain-specific datasets. A user-friendly web interface allows for seamless uploading of answer images, configuration of model settings, and visualization of results. Additionally, the system integrates a secure user authentication module for access control, enabling personalized model training and usage history tracking. Experimental results demonstrate that the fine-tuned SBERT model significantly improves semantic alignment with ground-truth answers, especially in the context of varied handwriting styles and non-standard grammar.

Introduction

The research presents an automated system for evaluating handwritten subjective answers using deep learning and natural language processing (NLP). It addresses common issues in manual grading such as bias, inconsistency, and delays. The system operates in two main steps:

1. Handwriting Recognition: Uses Google Cloud Vision API for Optical Character Recognition (OCR) to convert handwritten answers into machine-readable text, followed by spell correction to improve accuracy.

2. Semantic Similarity Evaluation: Compares the processed student answer with a model answer using advanced NLP techniques, primarily Sentence-BERT (SBERT), to assess semantic meaning rather than just keyword matching. Additional fuzzy matching and token-level comparison enhance robustness by capturing lexical similarities.

The system combines these methods into a weighted similarity score to assign marks and provide constructive feedback, enabling fair, objective, and efficient grading.

The literature survey covers various related methods in OCR, NLP, deep learning architectures, and applications for handwriting recognition and subjective answer evaluation, highlighting advances and challenges in the field.

The proposed system workflow includes image/text input, OCR processing, text preprocessing, similarity calculation (50% semantic similarity via SBERT, 30% fuzzy matching, 20% token matching), scoring, and feedback generation. It features a user-friendly web interface with options for uploading answers, model fine-tuning, and secure user authentication.

Results demonstrate that the system effectively automates grading with high consistency and provides detailed feedback, reducing manual workload and enhancing fairness in educational assessments.

Conclusion

Our proposed system significantly improves upon the base paper by addressing key limitations in handwritten subjective answer evaluation. First, it goes beyond basic character recognition by incorporating semantic understanding to evaluate the meaning and context of answers. Second, it includes spelling correction to enhance the accuracy of text processing. Third, it mitigates OCR related errors through the use of multiple text matching techniques. Additionally, the system eliminates manual grading bias by automating the evaluation process, ensuring fairness and consistency. Finally, it enhances user accessibility with a user-friendly interface that allows seamless answer uploads and result visualization. These advancements collectively make the system accurate and effective.

References

[1] Desai, Madhavi B., et al. \"A Survey on Automatic Subjective Answer Evaluation.\" Advances and Applications in Mathematical Sciences 20.11 (2021): 2749-2765. [2] Singh, Shreya, et al. \"Tool for evaluating subjective answers using AI (TESA).\" 2021 International Conference on Communication information and Computing Technology (ICCICT). IEEE, 2021. [3] Islam, Muhammad Nazrul, et al. \"A multilingual handwriting learning system for visually impaired people.\" IEEE Access (2024). [4] Rahaman, Md Afzalur, and Hasan Mahmud. \"Automated evaluation of handwritten answer script using deep learning approach.\" Transactions on Machine Learning and Artificial Intelligence 10.4 (2022). [5] Shaikh, Eman, et al. \"Automated grading for handwritten answer sheets using convolutional neural networks.\" 2019 2nd International conference on new trends in computing sciences (ICTCS). IEEE, 2019. [6] Nurseitov, Daniyar, et al. \"Classification of handwritten names of cities and handwritten text recognition using various deep learning models.\" arXiv preprint arXiv:2102.04816 (2021). [7] Rowtula, Vijay, Subba Reddy Oota, and C. V. Jawahar. \"Towards automated evaluation of handwritten assessments.\" 2019 International Conference on Document Analysis and Recognition (ICDAR). IEEE, 2019. [8] Sampathkumar, S. \"Kannada Handwritten Answer Recognition using Machine Learning Approach.\" (2022). [9] Kumar, Munish, et al. \"Character and numeral recognition for non-Indic and Indic scripts: a survey.\" Artificial Intelligence Review 52.4 (2019): 2235-2261. [10] Mukhopadhyay, Anirban, et al. \"A study of different classifier combination approaches for handwritten Indic Script Recognition.\" Journal of Imaging 4.2 (2018): 39. [11] Souibgui, Mohamed Ali, et al. \"Docentr: An end-to-(ICPR). IEEE, 2022. [12] Acharya, Shailesh, et al. \"Deep Learning Based Large Scale Handwritten Devanagari Character Recognition.\" [13] Ali, Amani Ali Ahmed, and Suresha Mallaiah. \"Intelligent handwritten recognition using hybrid CNN Journal of King Saud University-Computer and Information Sciences 34.6 (2022): 32943300. [14] Teslya, Nikolay, and Samah Mohammed. \"Deep Open Innovations Association (FRUCT). IEEE, 2022. [15] Alrobah, Naseem, and Saleh Albahli. “A Hybrid Deep Model for Recognizing Arabic Handwritten Characters”.

Copyright

Copyright © 2025 Dr. Dipak D. Bage, Tina H. Deore, Samarth S. Abak, Shruti D. Godse, Vishakha P. Mandawade, Dr. Aniruddha S. Rumale. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.