EDUSIGN - AI Powered Application for Deaf and Mute Students

Abstract

EDUSIGN is an artificial intelligence – powered application developed to assist deaf and mute students by improving communication and learning accessibility. The project focuses on recognizing sign language gestures using computervisionandmachinelearningtechniquesandconvertingthemintomeaningfultextandspeechinrealtime.The system captures hand gestures through a camera, processes them using trained AI models, and provides immediate outputthat canbeeasily understoodby hearing individuals.By enabling seamlessinteractionbetweenhearing-impaired students and the general population, EDUSIGN promotes inclusive education and equal learning opportunities. The application is designed to be user-friendly, efficient, and cost-effective, making it suitable for educational institutionsand real-world environments.

Introduction

The text presents EDUSIGN, an AI-based educational system designed to overcome communication barriers faced by deaf and mute students in traditional classrooms. Conventional teaching methods rely heavily on spoken language, limiting real-time interaction and active participation for hearing- and speech-impaired learners. EDUSIGN addresses this challenge by using artificial intelligence and computer vision to recognize sign language gestures and convert them into text and speech, enabling seamless two-way communication without requiring others to know sign language.

The primary objective of EDUSIGN is to promote inclusive education by allowing effective, real-time interaction between students, teachers, and peers. The system captures sign language through a camera, processes gestures using trained machine learning models, and generates appropriate responses. These responses are delivered via animated AI avatars that display Indian Sign Language (ISL), supporting both sign-to-text and text-to-sign communication.

EDUSIGN consists of several modules, including secure user authentication, a student dashboard for tracking lessons and progress, real-time sign-to-text recognition, and text-to-sign conversion using avatars. Teachers and administrators manage learning content, quizzes, and student performance through an admin dashboard. The platform supports interactive learning with instant AI-based feedback, quizzes, and progress tracking.

The system’s implementation uses OpenCV for hand gesture detection, TensorFlow-based models for gesture recognition, and text-to-speech technology for audio output. Data is securely stored to enable personalization and performance evaluation. Overall, EDUSIGN provides an accessible, interactive, and intelligent learning environment that enhances communication, independence, and academic participation for hearing- and speech-impaired students.

Conclusion

EDUSIGN is an innovative AI-powered application that aims to make learning and communication easier for deaf and mute students. By using artificial intelligence and computer vision, the system can recognize sign language gestures in real time and convert them into text and speech, helping students interact more effectively with teachersand peers. This project demonstrates how technology can bridge communication gaps and promote inclusive education. EDUSIGN allows hearing-impaired students to participate confidently in classroom activities and social interactions, reducing their reliance on human interpreters. With further development and integration of advanced AI and deep learning techniques, EDUSIGN has the potential to become a comprehensive assistive tool, making education more accessible and empowering students to learn and communicate without barriers.

References

[1] ElvinLalsiembul Hmar,BornaliGogoi, NelsonR.Varte, AComprehensiveSurveyonSign Language Recognition:Advances, Techniquesand Applications, International Journal of Engineering Research & Technology (IJERT), Vol. 14, Issue 08, Aug 2025. [2] Visiontransformer-poweredconversationalagentforreal-timeIndianSignLanguagee-governanceaccessibility,ScientificReports,19Apr2025. [3] Manasa Krishna BA et al., Connecting Worlds: A Deep Learning Approach to Real-Time Sign Language Translation, International Journal of EngineeringResearch & Technology (IJERT), Vol. 14, Issue 03, Mar 2025. [4] Mohammed AbdulKaderet al., Sign LanguageRecognition Based Communication System Using Machine Learning AlgorithmforVocally Impaired People,European Journal of Artificial Intelligence and Machine Learning. [5] Lavanya N L et al., Real–Time Sign Language Recognition and Multilingual Speech Output Based on Machine Learning, International Journal of HumanComputations and Intelligence (IJHCI), 2025. [6] Indian Sign Language (ISL) Translator: AI-Powered Bidirectional Translation System, IJRASET Journal for Research in Applied Science and EngineeringTechnology,2025. [7] RupeshKumar,AshutoshBajpai,AyushSinha, MediapipeandCNNsforReal-TimeASLGestureRecognition,arXivpreprint,2023. [8] Franco Ronchetti et al., Sign Language Recognition Without Frame-Sequencing Constraints: A Proof of Concept on the Argentinian Sign Language, arXivpreprint,2023. [9] LokeshRajput,MohammadAaqib,MeesalaSudhirKumar,ExploringtheImpactofAIonInclusiveEducation:CaseStudieswithSignLanguageandSpeech Therapy, IJARSCT, Vol. 4, Issue 5, Nov 2024.

Copyright

Copyright © 2026 P. Thilakaveni, M. Keerthana, M. Tejes, T. Mugilan, K. Yaswanth, P. Pavithran, K. Shri Shaarukesh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.