AI Virtual Mouse

Abstract

The AI Virtual Mouse is a revolutionary touchless replacement to standard computer mice that prioritizes hygiene and simplicity in an increasingly health-conscious world. Even while wireless mice do away with cable-related problems, they still require physical contact, which might spread germs, especially in offices where workstations are shared. By using an external or built-in camera on a computer to record and analyze hand and finger movements in real time, this state-of-the-art technology leverages computer vision and machine learning to do away with the need for touch. The AI Virtual Mouse accurately recognizes hand features using deep neural networks, allowing it to perform basic mouse activities including clicking, scrolling, and pointer navigation without requiring human interaction.This idea, which uses OpenCV in Python, enhances hygiene by significantly reducing the risk of contamination linked to high-touch surfaces while simultaneously improving user experience by doing away with the requirement for physical peripherals.

Introduction

The paper presents an AI-powered virtual mouse system that replaces traditional physical mice with hand gestures detected via a computer’s built-in or external camera. Using computer vision and machine learning (MediaPipe, OpenCV, PyAutoGUI, etc.), the system tracks hand landmarks and fingertip movements to perform mouse functions such as clicking, scrolling, and cursor movement without any physical contact or extra hardware like USB receivers or batteries.

This touchless interface offers a hygienic, convenient, and accessible alternative to conventional mice, especially useful in contexts like the COVID-19 pandemic or for users with hand impairments. The system processes real-time video input to detect raised fingers and interpret gestures, translating them into corresponding mouse actions. Various gestures control volume, scrolling, and clicking, with smoothing techniques applied for precise cursor control.

Compared to previous approaches that used gloves, color tracking, or complex hardware, this solution improves accuracy, responsiveness, and ease of use without requiring powerful GPUs. It enhances human-computer interaction (HCI) and has applications in VR/AR gaming, robotics, virtual prototyping, and more. The virtual mouse demonstrates how AI and computer vision can transform how users interact with computers through natural, hands-free gestures.

Conclusion

Currently,therearesomelimitationsthattheAIvirtualmouse technologyface.Italsohasatinydecreaseintheaccuracyof right- clicking, which may at times affect the precision of particular functions. Moreover, the modelstruggles withclick and drag operations—like text selection—which negatively affects work involving continuous drags. Such problems constitute some of the key restricts connected with Intelligencedigitalmice,andtheywillserveasinspirationfor future advances in our research. The system can also be further developed in future so as to mimic the keyboard and mouse both virtually. This could create new avenues in Human Computer Interaction(HCI).It represents the best potential for future development and use. The use of powerful learning techniques will also improve gesture identification, ensuring that accuracy does not suffer in low light or dense backdrops. Adding further platform support, such as macOS, Linux, and mobile compatibility, would increase the system\'s adaptability and make itavailableto alargerrangeofusers. Furthermore,the applicationofsuchtechnologiesinARandVRcontextsmay result in an extra user-friendly and captivating interface, particularly when it comes to gaming or cooperative virtual workplaces. Voice commands and gestures that may be customized for persons with impairments are examples of upcoming accessibility capabilities.The digital cursor has potential uses in industrial technology androbotics,allowingfreeofchargeandfastoperationusing gesture- based inputs. A Computer Vision Digital Mice via thePythonlanguageandOpenCVisaprogressininteraction between humans and computers who avoids contact with input devices and makes certain that there is a means of controlling button motion and go buttons to the application. Using easily accessible gear, such cameras and computer vision,thisprojectshowshowsimpleanduser-friendlyitcan be. The virtual mouse addresses hygiene concerns and can especiallyhelpfulinsituations wherehardwarecontact must be avoided,suchasduringa pandemic.Althoughthe system is currently only has basics cursor capabilities, this provides uswithasolidfoundationuponwhichtoexpandinthefuture.

References

[1] D. Lee, D.-H. Liou, and C.-C. Hsieh, \"A system for recognizing hand gestures in real time through the use of motion history images,\" in Proc. 2nd Int. Conf. Signal Process. Syst., IEEE, 2010 [2] D. L. Quam, \"The use of a Data Glove for gesture recognition,\"inProc.IEEESymp.AircraftTechnol.,vol.2,pp.755–760,1990. [3] D.-H. Liou, D. Lee, and C.-C. Hsieh, \"A live hand position detection device using motion history shots,\" in Proc.2ndWorldConf.Inf.Process.Syst.,vol.2,no.V2-394, IEEE, July 2010. [4] S. Thakur, \"Computing navigation using fingers and optics,\" IEEE Study, 2015. [5] S.P.Gopnarayan,G.Moroliya,andS.Patweka,\"Virtual mousethroughhandgesture,\"J.Emerg.Technol.Innov.Res., 2018. [6] N.Patil,G.Sali,andN.Lokhande,\"MousewithMLand AI on fingertips,\" Int. Res. Eng. Technol. J., 2019. [7] G.R.Fan,Z.Zhang,andV.Bazarevsky,\"TheMediaPipe for right now finger detection on devices,\" Oct. 9, 2023. [8] V.Kornyakovand M.Eruhimov,C.K.-M.A.Pulli,andB. Baksheev, \"Instantaneous speeds visual recognition with OpenCV,\" Queue, vol. 10, no. 4, pp. 40–56, 2012. [9] N.Asokkumar,S.Poddar,A.Haria,A. Subramanian, andJ.S.Nayak,\"Synopsisfortheidentificationofmovementsof the hand in interactions between humans and machines,\" ProcediaComput. Sci., vol. 115, pp. 367–374, 2017. [10] D. Shibly, D. Islam, and S. Showrav, \"Recognition of abstract hand gestures, virtual mouse, photo processing,\" in Proc. 1st Int. Conf. Adv. Robot. Sci. Eng. Technol. (ICASERT), Dhaka, Bangladesh, 2019, pp. 1–5, doi:10.1109/ICASERT. [11] K. P.Vinay,\"Holdaction-orientedpointercommand,\"Int.Encycl.ForensicAccount.Data,no.0975–8887,2016. [12] J. P. Katona, \"A review of the study domains of interaction between people and computers and virtual realities in neural interaction,\" Rec. Sci., vol. 11, no. 6, p. 2646, 2021. [13] S. U. Dudhane, \"Hand gesture detection-based cursor control system,\" Int. J. Adv. Res. Comput. Commun. Eng. (IJARCCE), vol. 2, no. 5, 2013.

Copyright

Copyright © 2025 Anuj Malik, Kartik Sharma, Rohit Negi, Joney Kumar, Punit Mittal, Dr. Swati Sharma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.