AI-Powered Multi-Functional Vision Assistant System for Visually Impaired

Abstract

Visually impaired individuals face significant challenges in safe navigation, obstacle avoidance, and real-time environmental awareness, which restrict their independence and mobility in daily life. Traditional assistive tools such as the white cane primarily rely on tactile feedback and are limited in detecting distant obstacles, surface hazards, and dynamic environmental conditions. To overcome these limitations, this paper presents an AI-powered multi-functional vision assistant designed to enhance safety, situational awareness, and autonomous navigation. The proposed system integrates ultrasonic sensing, AI-based hazard detection, GSM communication, and solar-powered energy management into a compact and cost-effective device. An Arduino Mega microcontroller coordinates system operations, while an ESP32- CAM module enables vision-based analysis using an Edge Impulse-based FOMO (Faster Objects, More Objects) model for real-time water hazard detection. Multiple ultrasonic sensors provide approximately 180° obstacle coverage, ensuring reliable detection with feedback delivered through audio or vibration alerts. The integration of a GSM module enables emergency communication and location sharing during critical situations. A solar-powered rechargeable system ensures continuous operation without dependence on external power sources. The system is designed to be lightweight, portable, and easy to use, making it suitable for real-world deployment. Furthermore, its low-cost implementation ensures accessibility for a wider population of visually impaired users. Experimental results demonstrate effective real-time performance, high detection accuracy, and robust operation across various conditions, making the proposed system a reliable, affordable, and user-friendly assistive solution for visually impaired individuals.

Introduction

This text presents an AI-Powered Multi-Functional Vision Assistant, a smart navigation aid designed to improve the mobility, safety, and independence of visually impaired individuals. While the traditional white cane remains the most common mobility tool, it only detects obstacles upon physical contact and cannot identify hazards such as water puddles, moving objects, or distant obstacles. Advances in embedded systems, sensors, and artificial intelligence have enabled the development of more intelligent assistive devices, but many existing solutions still suffer from limited functionality, high power consumption, short battery life, and lack of emergency communication features.

Background and Literature Review

Previous research has explored various smart walking sticks using technologies such as:

• Ultrasonic sensors for obstacle detection.
• Water sensors for detecting puddles and wet surfaces.
• GPS and GSM modules for navigation and emergency communication.
• Raspberry Pi and Arduino platforms for processing sensor data.
• Computer vision and AI models, including CNNs and YOLO, for object recognition and hazard detection.
• IoT and smartphone integration for navigation assistance and remote monitoring.

Studies have demonstrated that smart assistive devices can significantly improve user safety and independence through obstacle detection, object recognition, voice guidance, vibration alerts, and emergency communication. However, many systems either lack advanced hazard classification, consume excessive power, or remain too expensive for widespread adoption.

Proposed System

The proposed vision assistant addresses these limitations through a dual-controller architecture:

1. Arduino Mega
   • Acts as the main controller.
   • Coordinates sensors and system operations.
2. ESP32-CAM
   • Performs AI-based image processing.
   • Detects water hazards using machine learning.

Key Features

1. Multi-Directional Obstacle Detection

• Uses three ultrasonic sensors positioned to provide approximately 180° coverage.
• Detects obstacles on the left, center, and right sides.
• Provides immediate alerts through a buzzer and directional feedback.

2. AI-Based Water Hazard Detection

• Uses an ESP32-CAM and the FOMO (Faster Objects, More Objects) machine learning model developed with Edge Impulse.
• Identifies water puddles and slippery surfaces that are difficult to detect with conventional sensors.
• Warns users before they encounter hazardous areas.

3. Emergency Communication System

• Integrates a GSM module.
• Allows users to send emergency messages or calls to predefined contacts by pressing a button.
• Enhances safety during emergencies.

4. Solar-Powered Operation

• Includes a solar panel, rechargeable battery, and charge controller.
• Provides sustainable and portable power.
• Reduces dependence on external charging sources and extends operating time.

Advantages

Compared with traditional white canes and many existing smart sticks, the proposed system offers:

• Early obstacle detection before physical contact.
• AI-based recognition of water hazards.
• Wider environmental awareness through 180° sensing.
• Emergency communication capabilities.
• Renewable solar-powered operation.
• Improved safety, independence, and mobility for visually impaired users.
• Cost-effective and portable design.

Conclusion

The proposed AI-powered multi-functional vision assistant effectively addresses the critical challenges of safe navigation and environmental awareness for visually impaired individuals. By integrating multi-directional ultrasonic sensing, ESP32-CAM vision, GSM-enabled emergency communication, and a solar-powered energy system, the solution provides a comprehensive assistive tool that significantly enhances user independence and confidence. The system is designed to operate reliably in diverse real world environments, ensuring consistent performance under varying conditions. A fundamental strength of the system lies in its hybrid dual-controller architecture, which creates a strategic division of labor: the Arduino Mega facilitates real- time sensor coordination for immediate obstacle avoidance, while the ESP32-CAM performs edge-based intelligent hazard detection. This specific design achieves an optimal balance between computational efficiency, hardware cost, and real-time performance. The implementation of the Edge Impulse FOMO model enables the reliable identification of water puddles and slippery surfaces hazards that are notoriously difficult to detect using conventional moisture or ultrasonic sensors alone. Simultaneously, the 180? ultrasonic sensor arrangement provides wide-area spatial coverage, offering proactive alerts that allow for navigational corrections before physical contact occurs. Beyond mobility, the system prioritizes personal safety through a GSM module that enables immediate emergency triggers, while the incorporation of a solar- powered energy system ensures uninterrupted operation and reduces dependency on stationary charging infrastructure. Experimental evaluations confirm that the device is a lightweight, reliable, and cost-effective alternative to exist- ing assistive technologies, making it particularly suitable for cost-sensitive regions. This research represents a significant advancement in the field by successfully unifying embedded artificial intelligence and renewable energy into a practical, unified solution. Future work will focus on expanding the system’s capabilities through advanced object recognition for urban navigation, GPS-based tracking for real-time location sharing, and more intuitive voice-assisted interaction to pro- vide a richer, more descriptive understanding of the user’s environment.

References

[1] Roshan Roy K, Thanmay K Babu, Midhun E S, Arun A A, and Gadha M M, “A review on implementation of a multi-functional vision assistant for the visually impaired,” International Journal for Research in Applied Science & Engineering Technology (IJRASET), vol. 13, no. 12, pp. 3279–3287, Dec. 2025. [2] Siddharth Bhorge, Manas Kasodekar, Apoorv Karmalkar, and Vedant Kulkarni, “Smart blind stick: Object detection & GPS integration for enhanced mobility,” in Proceedings of the 3rd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), IEEE, 2024, pp. 1653–1658. [3] Dada Emmanuel Gbenga, Arhyel Ibrahim Shani, and Adebimpe Lateef Adekunle, “Smart Walking Stick for Visually Impaired People Using Ultrasonic Sensors and Arduino,” International Journal of Engineering and Technology (IJET), vol. 9, no. 5, pp. 3435–3447, Oct–Nov 2017. [4] Shahzor Memon, Mirza Muhammad Aamir, Sadiq Ur Rehman, Halar Mustafa, and Muhammad Shakir Sheikh, “Enhanced Mobility Aid for the Visually Impaired: An Ultrasonic Sensor and Arduino-Based Smart Walking Stick,” Memoria Investigaciones en Ingenier´?a, no. 28, pp. 20– 31, 2025. [5] Vidya M Shinde, Shivanjali Ninad Pawar, Nandinee Anil Wangane, Sampoorna Sampat Salve, and Swapnali Rajendra Wandhekar, “Smart assistive stick for visually impaired people,” World Journal of Advanced Engineering Technology and Sciences, vol. 14, no. 2, pp. 110–116, 2025. [6] Dipta Paul, S M Aliuzzaman, MD. Meraj Ali, Abatesham Rabbi, Md Fahim Khan, Niamul Alam, MD Tanvir Shakil, Dewan Saiful Islam, and Md Ariful Azad, “AI-Enhanced Multifunctional Smart Assistive Stick for Enhanced Mobility and Safety of the Visually Impaired,” Journal of Computer Science and Technology Studies, vol. 7, no. 1, pp. 283–301, 2025. [7] Ahmet Karagoz and Gokhan Dindis, “Object Recognition and Po- sitioning with Neural Networks: Single Ultrasonic Sensor Scanning Approach,” Sensors, vol. 25, no. 1086, 2025. [8] K. Ramarethinam, Mrs K. Thenkumari, and P. Kalaiselvan, ”Navigation System for Blind People Using GPS & GSM Techniques”, International Journal of Advanced Research in Electrical, Electronics and Instrumen- tation Engineering, vol. 3, no. Special Issue 2, pp. 399–405, Apr. 2014. [9] S. Singh and Dr. B. Singh, “Intelligent Walking Stick for Elderly and Blind People”, International Journal of Engineering Research & Technology (IJERT), vol. 9, no. 03, pp. 19–22, Mar. 2020. [10] T. Tirupal, B. Venkata Murali, M. Sandeep, K. Sunil Kumar, and C. Uday Kumar, “Smart Blind Stick Using Ultrasonic Sensor,” Journal of Remote Sensing GIS & Technology, vol. 7, no. 2, pp. 34–42, May–Aug. 2021. [11] L. D´?az-Vilar´?no, P. Boguslawski, K. Khoshelham, H. Lorenzo, and L. Mahdjoubi, “Indoor Navigation from Point Clouds: 3D Modelling and Obstacle Detection,” The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL-B4, pp. 275– 281, 2016. [12] P. S, P. S. Nihal, R. R. Menon, S. S. Kumar, and S. Tiwari, “Smart Blind Stick using Artificial Intelligence,” International Journal of Engineering and Advanced Technology (IJEAT), vol. 8, no. SS, pp. 19–22, May 2019. [13] Laurence Kenneth A. Balomaga, Dr. Dan Michael A. Cortez, Charmaine Allyson P. Reyes, Criselle J. Centeno, Raymund M. Dioses, and Leisyl [14] M. Mahusay, “ProxiSense: IoT Smart Blind Stick with Voice Alerts for Obstacle and Water Hazard Detection,” 2024 IEEE Conference on Innovative Data Communication Systems and Engineering (ICIDCSE), pp. 1–7, 2024. DOI: 10.1109/CISES63760.2024.1091035. [15] N. Loganathan, K. Lakshmi, N. Chandrasekaran, S. R. Cibisakar- avarthi, R. Hari Priyanga, and K. HarshaVarthini, “Smart Stick for Blind People,” 2020 6th International Conference on Advanced Com- puting & Communication Systems (ICACCS), pp. 65–67, 2020. DOI: 10.1109/ICACCS48705.2020.9074374. [16] Mrs. K.P. Kamble, Shambhavi Shende, Ved Sahu, Karan Bariya, and Gargi Deosthali, “Voice Assisted Smart Blind Stick,” 2024 5th Inter- national Conference on Electronics and Sustainable Communication Systems (ICESC), pp. 1565–1569, 2024. [17] Romteera Khlaikhayai, Chavana Pavaganun, Benja Mangalabruks, and Preecha Yupapin, “An Intelligent Walking Stick for Elderly and Blind Safety Protection,” Procedia Engineering, vol. 8, pp. 313–316, 2011. DOI: 10.1016/j.proeng.2011.03.058. [18] Ahmed Ben Atttallah, Yahia Said, Mohamed Amin Ben Atttallah, Mohammed Albekairi, Khaled Kaaniche, and Sahbi Boubaker, “An effective obstacle detection system using deep learning advantages to aid blind and visually impaired navigation,” Ain Shams Engineering Journal, vol. 15, p. 102387, 2024. DOI: 10.1016/j.asej.2023.102387. [19] Ammar Almomani, Mohammad Alauthman, Amal Malkawi, Hadeel Shwaihet, Batool Aldigide, Donia Aldabeck, and Karmen Abu Hamoodeh, “Smart Shoes Safety System for the Blind People Based on (IoT) Technology,” CMC, vol. 76, no. 1, pp. 415-434, 2023. DOI: 10.32604/cmc.2023.036266. [20] Thayer Corbin, “Vision-Based Autonomous Navigation and Obstacle Avoidance in Mobile Robots Using Deep Reinforcement Learning,” 2023. [21] Mustufa Haider Abidi, Arshad Noor Siddiquee, Hisham Alkhalefah, and Vishwaraj Srivastava, “A comprehensive review of navigation systems for visually impaired individuals,” Heliyon, vol. 10, no. 10, p. e31825, 2024. DOI: 10.1016/j.heliyon.2024.e31825. [22] Sangam Malla, Prabhat Kumar Sahu, Srikanta Patnaik, and Anil Kumar Biswal, “Obstacle Detection and Assistance for Visually Impaired Indi- viduals Using an IoT-Enabled Smart Blind Stick,” Revue d’Intelligence Artificielle, vol. 37, no. 3, pp. 783-794, 2023. DOI: 10.18280/ria.370327. [23] P. Vennila, V. Alamelu Mangayarkarasi, K. Vinayakan, and G. R. Gnana Raja, “Smart IoT Navigation System for Visually Impaired Individuals: Improving Safety and Independence with Advanced Obstacle Detection,” International Journal of Computational Research and Development, vol. 10, no. 1, pp. 17-23, 2025.

Copyright

Copyright © 2026 Roshan Roy K, Midhun E S, Thanmay K Babu, Arun A A, Gadha M M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.