DESKXI: Design and Development of a Voice-First Desktop AI Mentor for Interactive Learning, Personalized Study Assistance, and Intelligent Task Management

Abstract

Students who study alone rarely have a tutor, an interview coach and a study planner within reach, and screen-based assistants pull attention away from the work itself. DESKXI is a voice-first desktop AI mentor that combines these roles in one device: it teaches by conversation, runs mock interviews, sets spoken quizzes, keeps Pomodoro study cycles, delivers reminders and retains durable facts about the learner between sessions, all through a streaming cloud speech-to-speech model. This paper describes how DESKXI divides work between the cloud model, which carries the open-ended conversation, and deterministic local services, which hold exact state such as timers, reminders and quiz keys. It also describes the real-time client runtime that carries these functions: a bounded drop-oldest capture queue, a writer-thread playback path, a microphone mute gate, a three-tier recovery design and a SQLite store. We examine selected parts of this runtime by running the unmodified source code on a reference workstation with only the hardware and cloud modules stubbed. The measurements identified four limitations that matter for mentoring: speech that keeps playing for about 1.1 s after the learner interrupts, an unfiltered down-sampler that folds out-of-band energy into the speech band, reminders that fire up to 0.95 s early, and a quiz-answer parser that reached 44.6 % accuracy on our author-written stress set. We give a remedy for each. Learning outcomes were not studied.

Introduction

DESKXI is a voice-based AI study assistant designed to help students learn independently without constantly interacting with a screen. It combines the roles of a tutor, practice partner, and study organiser in a single desktop device. The system uses a cloud-based speech-to-speech AI model for natural conversation and local software services for tasks that require accurate timing, persistent storage, and deterministic results.

The system provides several features, including teaching mode, mock interviews, spoken multiple-choice quizzes, Pomodoro study sessions, timed reminders, weather and web searches, and long-term memory of learner information. It uses a USB microphone and Bluetooth speaker to support hands-free interaction.

The architecture separates conversational intelligence from essential local functions. The cloud model handles explanations, questions, and feedback, while local Python services and an SQLite database manage quiz answers, study timers, reminders, and stored information. This approach helps ensure that important functions do not depend entirely on the language model. A three-tier recovery mechanism reconnects interrupted sessions, saves conversation data, and restarts the application after unexpected failures.

The research focuses on the reliability and real-time performance of the system rather than its educational effectiveness. It analyses audio buffering, interruption handling, reminder timing, audio down-sampling, and recovery mechanisms. Experiments use the actual application code with hardware and cloud services replaced by test components.

The reported findings reveal several limitations. First, audio may continue playing for approximately 1.0–1.2 seconds after a user interrupts the assistant, causing overlapping speech and reducing conversational responsiveness. A smaller audio pipe significantly reduces this delay. Second, the bounded microphone queue prevents excessive audio delays but may discard audio during network stalls. Third, the reminder system can deliver notifications more than once if the application restarts at a particular point, and reminders tend to occur slightly earlier than scheduled. Additionally, active quiz questions are lost when the application restarts, while timers and reminders are stored persistently in SQLite.

The study also examines audio down-sampling and the performance of local services. However, the reported tests were conducted on a reference workstation rather than the intended single-board computer, so the results may differ on the actual device.

In conclusion, DESKXI presents a practical architecture for a hands-free AI study mentor by integrating conversational AI with reliable local services. Its main contribution is the examination of real-time audio processing, interruption handling, persistent storage, and automatic recovery. Although the system offers a broad set of mentoring features, further improvements are needed in interruption responsiveness, audio quality, reminder reliability, and state persistence. The research does not claim improved learning outcomes or student satisfaction, as these require separate controlled user studies.

Conclusion

DESKXI assigns open-ended teaching and interview talk to a cloud speech model and exact state to local services. A bounded drop-oldest capture queue, a writer thread for playback and a restart-based supervisor over a durable SQLite store form a runtime designed to keep capture latency bounded and to preserve study state. Workstation measurements of selected parts of that runtime exposed four limitations: speech that outlasts an interruption, an unfiltered down-sampler, early reminders and a fragile quiz-answer parser. The method needs no robot hardware, only the code, and it found defects that conversation tests would not. Next steps are replication on the target board, evaluation of the capture, reconnect and restart mechanisms there, a gate that tracks playback, and a user study of whether the improved runtime changes how people judge the robot.

References

[1] G. Skantze, “Turn-taking in conversational systems and human–robot interaction: A review,” Comput. Speech Lang., vol. 67, Art. no. 101178, 2021. [2] T. Stivers et al., “Universals and cultural variation in turn-taking in conversation,” Proc. Natl. Acad. Sci. USA, vol. 106, no. 26, pp. 10587–10592, 2009. [3] ITU-T, “One-way transmission time,” Recommendation G.114, 2003. [4] C. Y. Kim, C. P. Lee, and B. Mutlu, “Understanding large-language model (LLM)-powered human-robot interaction,” in Proc. ACM/IEEE Int. Conf. Human-Robot Interaction, 2024. [5] A. Défossez et al., “Moshi: A speech-text foundation model for real-time dialogue,” arXiv:2410.00037, 2024. [6] Z. Ma et al., “Language model can listen while speaking,” in Proc. AAAI Conf. Artif. Intell., 2025, arXiv:2408.02622. [7] Google, “Live API capabilities guide,” Gemini API documentation. [Online]. Available: https://ai.google.dev/gemini-api/docs/live-api/capabilities (accessed Oct. 2026). [8] Google, “Live API session management,” Gemini API documentation. [Online]. Available: https://ai.google.dev/gemini-api/docs/live-api/session-management (accessed Oct. 2026). [9] Gemini Team, Google, “Gemini: A family of highly capable multimodal models,” arXiv:2312.11805, 2023. [10] G. Candea and A. Fox, “Crash-only software,” in Proc. 9th Workshop Hot Topics Operating Syst. (HotOS IX), 2003. [11] D. Patterson et al., “Recovery-oriented computing (ROC): Motivation, definition, techniques, and case studies,” Univ. California, Berkeley, Tech. Rep. UCB//CSD-02-1175, 2002. [12] J. Armstrong, “Making reliable distributed systems in the presence of software errors,” Ph.D. dissertation, Royal Inst. Technol. (KTH), Stockholm, Sweden, 2003. [13] J. Gettys and K. Nichols, “Bufferbloat: Dark buffers in the Internet,” Commun. ACM, vol. 55, no. 1, pp. 57–65, 2012. [14] J. Dean and L. A. Barroso, “The tail at scale,” Commun. ACM, vol. 56, no. 2, pp. 74–80, 2013. [15] R. E. Crochiere and L. R. Rabiner, Multirate Digital Signal Processing. Englewood Cliffs, NJ, USA: Prentice-Hall, 1983. [16] A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing, 3rd ed. Upper Saddle River, NJ, USA: Pearson, 2010. [17] J. Benesty et al., Advances in Network and Acoustic Echo Cancellation. Berlin, Germany: Springer, 2001. [18] SQLite Consortium, “Atomic commit in SQLite.” [Online]. Available: https://www.sqlite.org/atomiccommit.html (accessed Oct. 2026). [19] M. Kleppmann, Designing Data-Intensive Applications. Sebastopol, CA, USA: O’Reilly, 2017. [20] T. Belpaeme, J. Kennedy, A. Ramachandran, B. Scassellati, and F. Tanaka, “Social robots for education: A review,” Sci. Robot., vol. 3, no. 21, Art. no. eaat5954, 2018. [21] T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro, “Interactive robots as social partners and peer tutors for children: A field trial,” Hum.–Comput. Interact., vol. 19, no. 1–2, pp. 61–84, 2004.

Copyright

Copyright © 2026 Urmil Mamulkar, Deepak Kumar , Sumedh Dhoke , Vijay Sharma, Sejal Farkade , Shrushti Bhosale, Dr. Ruchi Kadwane. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.