Smart Voice Assistant with Face Recognition

Abstract

Individuals with disabilities often struggle to navigate conventional digital environments due to physical, visual, or auditory limitations. This study introduces a desktop-based intelligent assistant that merges voice control with facial recognition to deliver an inclusive, user-friendly solution. By harnessing the capabilities of Artificial Intelligence (AI), Computer Vision, and Natural Language Processing (NLP), the system offers a flexible, multi-modal interface. Voice interaction assists visually impaired users, text-based communication supports those with hearing impairments, and facial recognition aids users with limited physical mobility. The proposed solution emphasizes offline functionality, security, and accessibility, aiming to empower users through improved digital interaction and increased independence.

Introduction

In today’s digital world, technology significantly enhances communication and accessibility, but individuals with disabilities still face barriers when using conventional computing systems. Many existing assistive technologies—like voice assistants, screen readers, and gesture controls—fail to support users with multiple or complex impairments. This research introduces a desktop-based Smart Voice Assistant with Face Recognition designed to bridge these gaps through a multi-modal interface incorporating voice, text, and facial inputs.

Key Features of the Proposed System:

• Designed for Visually Impaired Users: Hands-free voice commands reduce reliance on screens.

• Support for Hearing-Impaired Users: Real-time text and sign language recognition enable communication without audio.

• Assistance for Physically Disabled Users: Facial recognition allows secure, hands-free access and interaction.

Technologies Used:

• Programming in Python

• Libraries/APIs: speech_recognition, pyttsx3, Vosk, OpenCV, NLTK, spaCy

• Offline capability for privacy and functionality in low-connectivity environments

• GUI built using PyQt and Tkinter

Methodology & Experimental Setup:

• Built as a Windows desktop application

• Tested with 15 users (with visual, auditory, and physical disabilities) over one week

• Evaluated under different lighting and noise conditions

• Assessed usability, performance, and accessibility

Results:

• Speech-to-text accuracy: 91% (moderate noise), 84% (high noise)

• Face recognition accuracy: 96% (good lighting), 88% (dim lighting)

• System Usability Scale (SUS) Score: 88 (high satisfaction)

• Offline functionality ensured user privacy and accessibility in varied conditions

Contributions:

• Provides a holistic, inclusive, and secure digital assistant for differently-abled users

• Combines AI, NLP, computer vision, and HCI to deliver personalized and accessible user experiences

• Promotes digital equity by supporting multiple modes of interaction and ensuring offline operation

Conclusion

The creation of a Voice Assistant integrated with Face Recognition represents a major advancement in assistive technology. By integrating AI, Natural Language Processing (NLP), and Computer Vision, it delivers a secure, multimodal desktop interface designed specifically for people with disabilities. This system accommodates voice, text, and facial recognition inputs, promoting hands-free accessibility and custom-tailored interactions. It enhances user autonomy in the digital world and stands as a prime example of ethical, usercentric AI development.

References

[1] 2020 13th CMI Conference on Cybersecurity and Privacy (CMI) [2] International Journal of Engineering Applied Sciences and Technology, 2022 Vol. 7 [3] IOSR Journal of Computer Engineering (IOSR-JCE), Volume 20 [4] International Journal of Advanced Research in Science Communication and Technology [5] Journal of Emerging Technologies and Innovative Research 11(6) [6] Palleti, V. (2021). SpeechRecognition. PyPI. [7] Yadav, M. (2021). pyttsx3. PyPI. [8] GitHub. (n.d.). Topics - desktop-assistant. [9] Nallamothu, M., & Mukkamala, R. (2019). A Study on Speech Recognition and Desktop Assistant using Python. Journal of Emerging Technologies and Innovative Research, 6(4), 521-526. [10] Mukherjee, A. (2019). Survey on Virtual Assistant (Google Assistant, Siri, Cortana, Alexa). 4th International Symposium SIRS 2018, Revised Selected Papers, 65-74. [11] Tripathy, S. (2020). Build Your Own Desktop Voice Assistant in Python. Analytics Vidhya. [12] Kumar, A., Kumar, A., & Kumar, P. (2021). Development of a Voice Assistant using Python. International Journal of Research in Engineering and Science, 10(2), 43-50. [13] Ms. Preethi G, Mr. Thiruppugal, Mr. Abhishek, & Mr. Vishwas D A “Voice Assistant using Artificial Intelligence ” IJERT 2022. [14] Faruk Lawal Ibrahim Dutsinma, Debajyoti Pal, Suree Funikul, & Jonathan H. Chan “A Systematic Review of Voice Assistant Usability: An ISO 9241-11 Approach”2022. Vishalkumar ”Research Paper on Desktop Voice Assistant” International Journal of Research in Engineering and Science, Volume 10 Issue 2, February

Copyright

Copyright © 2025 Raj Barot, Mahesh Panchal. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.