EYRO - AI Glasses: A Unified Multimodal Assistive Navigation Prototype

Abstract

EYRO is a prototype assistive-navigation system designed to combine visual perception, speech interaction, task management, and audio guidance for people with visual impairment. Unlike systems that concentrate on a single capability, the implementation integrates walkable-area segmentation, obstacle detection, crosswalk and traffic-light assistance, open-vocabulary object search, hand-guided localization, experimental Indian currency recognition, browser-based map navigation, and fail-safe health monitoring under one task-controlled application. The software runs primarily on a Windows laptop and accepts camera and microphone input from local devices or optional network-connected hardware. A NavigationMaster state machine coordinates the active task so that only one primary guidance workflow is executed at a time, while a reduced-rate obstacle safety layer can remain active during selected temporary tasks. The implementation uses SegFormer-B0 for walkable-area segmentation, YOLO11n for obstacle and traffic-related detection, YOLOE for open-vocabulary search and banknote localization, ByteTrack for object tracking, MediaPipe HandLandmarker for hand guidance, faster-whisper for speech recognition, pyttsx3/SAPI5 for speech output, OpenRouteService for map routing, and optional MobileNetV3-Small classifiers for currency denomination and authenticity estimation. The system deliberately avoids unverified safety claims: an unclear traffic light remains UNKNOWN, a nearby-vehicle estimate is treated as a visual proxy rather than distance or speed, and currency output is explicitly an AI estimate. This paper presents the engineering design, module-level methodology, state-machine logic, implementation decisions, comparison with a recent smart-glasses base paper, and an evaluation framework. Because project-specific benchmark measurements and controlled user-study results were not supplied in the source materials, no fabricated accuracy, latency, FPS, or user-performance values are reported.

Introduction

The text introduces EYRO, an integrated AI-based assistive system designed to support people with visual impairments through environmental perception, navigation, object recognition, and voice-based interaction. Unlike conventional assistive tools that focus on individual tasks, EYRO combines multiple functions within a single software architecture.

The selected 2026 base paper by Kumari and Hammady uses Vuzix Blade 2 smart glasses to integrate object detection, distance estimation, optical character recognition (OCR), and audio feedback. In contrast, EYRO uses a Windows laptop as its primary processing unit, with a camera, microphone, browser interface, FastAPI server, and Windows SAPI5 speech engine. This makes the prototype easier to develop and modify, although it is not yet a compact, standalone wearable device.

Key Features of EYRO

EYRO integrates several assistive modules:

• Blind Path Navigation: Uses SegFormer-B0, YOLO11n, and optical flow to identify walkable areas and obstacles and provide audio guidance.

• Crossing Assistance: Detects crosswalks, traffic lights, and vehicles to provide cautious crossing-related guidance.

• Traffic-Light Recognition: Uses object detection and region-of-interest colour analysis to classify signals as red, yellow, green, or unknown.

• Object Search: Combines YOLOE, ByteTrack, and MediaPipe HandLandmarker to locate, track, and guide users toward requested objects.

• Map Navigation: Uses browser geolocation and OpenRouteService to provide destination-based, turn-by-turn spoken directions.

• Currency Recognition: Estimates Indian banknote denominations and possible authenticity-related indicators while communicating uncertainty.

• Voice Interaction and Chat: Supports spoken commands, typed inputs, and conversational responses when no other task is active.

Main Research Problem

The primary challenge is not developing a completely new AI model, but integrating multiple assistive functions into one coordinated system while ensuring that the guidance remains relevant, consistent, and safe.

For example, when a user is walking, the system may need to provide obstacle warnings before temporarily switching to crossing assistance or object search. EYRO therefore includes a shared task manager that determines which task controls the main audio guidance channel, when temporary tasks end, and how the system returns to its default state.

The prototype also emphasizes conservative safety behaviour:

• Uncertain traffic-light detections are reported as UNKNOWN rather than forced into a definite category.

• Vehicle proximity is estimated using bounding-box size, not verified physical distance or speed.

• Currency predictions are treated as AI estimates, not guarantees of authenticity.

• Failed detections and outdated results must not produce misleading or stale guidance.

Research Gap and Contribution

Existing assistive technologies often concentrate on specific functions, such as obstacle detection, text reading, or navigation. Combining these functions introduces additional challenges involving task coordination, uncertainty management, audio prioritization, and failure handling.

EYRO addresses this gap through a modular software architecture that integrates diverse perception and interaction technologies under a common state machine. Its contribution lies primarily in system integration, task orchestration, and uncertainty-aware feedback, rather than claiming that its individual AI models are novel.

Evaluation and Limitations

The project aims to evaluate functionality, reliability, interaction behaviour, and performance in real-world environments. However, the text does not provide sufficient project-specific experimental results to establish overall accuracy, real-time performance, or safety effectiveness.

Important limitations include:

• Dependence on a laptop for primary processing.

• Potential sensitivity to lighting, environmental conditions, and detection errors.

• Dependence on external connectivity for some navigation and optional chat functions.

• Lack of certification as a medical or safety device.

• Need for further real-world testing before practical deployment.

References

[1] P. Kumari and R. Hammady, “Assisting blind people with AI and audio using smart glasses: system design with YOLOv8 variants comparisons,” Multimedia Systems, vol. 32, art. 73, 2026, [2] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 34, 2021. [3] Y. Zhang, P. Sun, Y. Jiang, D. Yu, F. Weng, Z. Yuan, P. Luo, W. Liu, and X. Wang, “ByteTrack: Multi-Object Tracking by Associating Every Detection Box,” in European Conference on Computer Vision (ECCV), 2022. [4] F. Zhang, V. Bazarevsky, A. Vakunov, A. Tkachenka, G. Sung, C.-L. Chang, and M. Grundmann, “MediaPipe Hands: On-device Real-time Hand Tracking,” arXiv:2006.10214, 2020. [5] A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, Q. V. Le, and H. Adam, “Searching for MobileNetV3,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1314–1324, 2019. [6] A. Radford et al., “Robust Speech Recognition via Large-Scale Weak Supervision,” in Proc. 40th International Conference on Machine Learning, PMLR 202, pp. 28492–28518, 2023. [7] A. Wang, L. Liu, H. Chen, Z. Lin, J. Han, and G. Ding, “YOLOE: Real-Time Seeing Anything,” arXiv:2503.07465, 2025. [8] G. Jocher and J. Qiu, Ultralytics YOLO11, Ultralytics, 2024. Official documentation and software release information. [9] openrouteservice, “Directions Service and API Documentation,” openrouteservice API documentation, accessed 2026. [10] M. Zöllner, S. Huber, H.-C. Jetter, and H. Reiterer, “NAVI—A proof-of-concept of a mobile navigational aid for visually impaired based on the Microsoft Kinect,” in INTERACT 2011, Part IV, Lisbon, Portugal, 2011.

Copyright

Copyright © 2026 Shravani Mahajan, Srushti Nitnaware, Ishant Hirekhan, Mayank Jorande , Ms. Priya Meshram. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.