Navi: RAG-Powered LLM Chatbot for Academic Institutions

Abstract

With the growing integration of artificial intelligence (AI) across educational ecosystems, there is an increasing demand for intelligent conversational agents that can efficiently deliver reliable, domain-specific information to students, faculty, and visitors. This research introduces Navi, an academic virtual assistant designed using a Large Language Model (LLM) combined with a Retrieval-Augmented Generation (RAG) framework to generate accurate and contextually grounded responses [1], [5]. The chatbot incorporates the Mistral-7B-Instruct model [3] for response generation and leverages a FAISS-based vector database [4], where embeddings are produced using the all-MiniLM-L6-v2 sentence transformer model. When a user submits a query, relevant document segments are retrieved from institutional data sources and integrated into the LLM’s prompt, enabling precise, factual, and contextually aligned output. Navi offers a range of advanced capabilities, including natural language understanding, multi-turn contextual dialogues, multilingual query handling, sentiment adaptation, speech-enabled interaction, and user-personalized responses. Performance evaluation through simulated academic queries indicates improved response accuracy, coherence, and informativeness, achieving an average relevance score between 0.7–0.85. The experimental results confirm that combining RAG with an LLM substantially reduces hallucinations, enhances factual grounding, and improves user satisfaction. Overall, Navi demonstrates a scalable and dependable framework for deploying AI-driven information assistants within educational institutions.

Introduction

The text discusses the development of Navi, an advanced RAG-powered LLM chatbot designed for academic institutions. It highlights the limitations of traditional rule-based chatbots and even standalone Large Language Models (LLMs), such as lack of contextual accuracy, outdated information, and hallucinations. Retrieval-Augmented Generation (RAG) is introduced as an effective solution that grounds responses in verified, domain-specific documents, improving factual reliability.

Navi integrates the Mistral-7B-Instruct LLM, FAISS vector database, and semantic embeddings to process institutional documents like syllabi, admission brochures, and faculty information. User queries are semantically matched to the vector store, and the retrieved text is appended to the prompt before generation, ensuring accurate and context-aware responses. The system also supports multilingual queries, multi-turn dialogue, sentiment-adaptive replies, and voice interaction.

A literature review shows the evolution of conversational AI—from rigid rule-based systems to LLMs, and now to RAG architectures that reduce hallucinations and improve transparency. Prior studies demonstrate the effectiveness of RAG in academic and enterprise applications.

The methodology covers data collection, chunking, embedding generation, database construction, retrieval pipeline implementation, and frontend design. Evaluation results show significant performance improvements after implementing RAG: retrieval accuracy increased from 78% to 92%, relevance scores improved, and hallucination rates dropped substantially. User satisfaction also rose due to more reliable and contextually grounded responses.

Despite strong performance, limitations include dependency on the completeness of institutional data and slower processing in CPU-only environments. Future enhancements may include GPU acceleration, hybrid retrieval, and continuous dataset updates.

Overall, Navi demonstrates that combining LLMs with RAG provides a scalable, accurate, and transparent conversational system suitable for real-world academic environments.

Conclusion

The experimental findings clearly establish that integrating Large Language Models (LLMs) with Retrieval-Augmented Generation (RAG) substantially enhances the accuracy, contextual understanding, and factual reliability of conversational AI systems in academic domains [1], [2], [19]. Compared to a standalone LLM, the proposed hybrid framework demonstrates notable improvements in retrieval accuracy, response relevance, and user satisfaction, confirming the practical advantages of grounding generative models in verified data sources [5], [7], [20]. From an application standpoint, Navi’s performance metrics highlight its potential as a real-time institutional assistant for college websites. The system effectively addresses frequent academic inquiries—ranging from admissions and course structures to administrative policies—while maintaining factual consistency. By leveraging FAISS-based semantic search and Sentence-BERT embeddings, the chatbot consistently retrieves the most relevant document segments, ensuring contextual precision and minimizing hallucinations [3], [6], [18]. Moreover, the framework demonstrates adaptability to multiple use cases beyond education. The modular architecture, built on Flask and FAISS, allows for seamless integration with existing information systems, thereby reducing administrative workload and enhancing accessibility for students, faculty, and visitors [1], [2], [17]. Its multilingual, sentiment-adaptive, and voice-interactive capabilities further improve user engagement, aligning with current trends in human-centered AI deployment [9], [12], [19]. Despite these achievements, several limitations were identified during evaluation. The system’s accuracy depends on the completeness and recency of institutional data available in the knowledge base. Queries extending beyond the stored corpus occasionally result in low-confidence responses or limited contextual depth [10], [19]. Additionally, the CPU-based deployment restricts processing speed for complex multi-turn interactions [3]. Future improvements could include GPU-accelerated inference, dynamic dataset updates, and hybrid retrieval models combining dense and sparse search mechanisms to further refine performance [8], [9], [11]. Beyond technical refinements, long-term evaluation focusing on user engagement trends, system adaptability, and satisfaction metrics will help assess the chatbot’s sustained effectiveness within real-world academic workflows [12], [15]. Expanding the model to handle multimodal content, such as images or PDFs containing tables, can further strengthen its value in academic administration and research support [11], [21]. In conclusion, the Navi: RAG-Powered LLM Chatbot for Academic Institutions represents a significant contribution toward building transparent, domain-specific, and scalable conversational systems. The combination of retrieval-based grounding and generative intelligence successfully mitigates hallucination while improving contextual fidelity. This study reaffirms that RAG-enhanced LLMs can serve as a cornerstone for deploying reliable AI assistants in educational environments and can be seamlessly extended to other sectors requiring structured, fact-driven information systems [1], [5], [19].

References

[1] The paper titled \"LLM and RAG Powered Chatbot for the College of Computer Science and Mathematics at the University of Mosul\", published in October-2024, was authored by Ban Sharief Mustafa, Yusuf Ersayyem Madhi, and presented in the International Research Journal of Innovations in Engineering and Technology (IRJIET) . [2] The paper titled \"RAG based Chatbot using LLMs \", published in 2020, was authored by Ananya G and Dr. Vanishree K, and presented in the International Journal of Scientific Research in Engineering and Management (IJSREM). [3] The paper titled \"Faiss: A library for efficient similarity search\", posted on march 29, 2017, was authored by Hervé Jegou, Matthijs Douze, Jeff Johnson., and presented in the Ai Research, Data Infastructure, ML Applications. [4] The paper titled \"LangChain: Building Applications with LLMs through Composability\", published in 2022, was authored by Harrison Chase and Josh Petterson, and presented in the ArXiv conference. [5] The paper titled \"Mistral 7B\", published in 2023, was authored by The Mistral AI Team, and presented in the Mistral AI Technical Report. [6] The paper titled \"FAISS: A Library for Efficient Similarity Search and Clustering of Dense Vectors\", published in 2017, was authored by Johnson, Douze, and Jégou, and presented in the Journal of Machine Learning Research (JMLR). [7] The paper titled \"REALM: Retrieval-Augmented Language Model Pre Training\", published in 2020, was authored by Kelvin Guu et al., and presented in the ICML conference. [8] The paper titled \"FiD: Fusion-in-Decoder for Open-Domain Question Answering\", published in 2021, was authored by Patrick Lewis et al., and presented in the ICLR conference. [9] The paper titled \"Dense Passage Retrieval for Open-Domain Question Answering\", published in 2020, was authored by Vladimir Karpukhin et al., and presented in the EMNLP conference. [10] The paper titled \"ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT\", published in 2020, was authored by Omar Khattab and Matei Zaharia, and presented in the SIGIR conference. [11] The paper titled \"Blended RAG: Improving RAG Accuracy with Semantic Search and Hybrid Query-Based Retrievers\", published in 2024, was authored by Kunal Sawarkar et al., and presented in the ArXiv repository. [12] The paper titled \"Reinforcement Learning for Optimizing RAG for Domain Chatbots\", published in 2024, was authored by Mandar Kulkarni et al., and presented in the ArXiv repository. [13] The paper titled “Implementation of a Chatbot System using AI and NLP”, published in May 2018, was authored byLalwani, Tarun and Bhalotia, Shashank and Pal, Ashish and Rathod, Vasundhara and Bisen, Shreya and presented in the International Journal of Innovative Research in Computer Science & Technology (IJIRCST) Volume-6,Issue-3,May-2018. [14] The paper titled “College Enquiry Chatbot”, published in Dec 2021, was authored by Shubhanshu Bhardwa, Snehil Khatri,Swati Khare and presented International Journal for Research in Applied Science & Engineering Technology (IJRASET)Volume 9 Issue XII Dec 2021. [15] The paper titled “Retrieval-Augmented Generation for Large Language Models: A Survey”, published in Mar 2024, was authored by Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang and presented International Journal for Research in Applied Science & Engineering Technology (IJRASET)Volume 9 Issue XII Dec 2021. [16] “Large language models struggle to learn long-tail knowledge,” published in 2023, was authored by N. Kandpal, H. Deng, A. Roberts, E. Wallace, and C. Raffel,and presented in International Conference onMachine Learning.PMLR,2023,pp.15696–15707. [17] The paper titled “Design of Chatbot System for College Website”, published in July 2021, was authored by Shivani Pravin Rashinkar, Neha Dilip Wanjol, Shivani Nandkumar Rane, Pushkar P. Shinde, Sopan A. Talekar and presented in International Journal of Computer Sciences and Engineering (JCSE) Volume 9, Issue-7,July 2021 E-ISSN: 2347-2693. [18] The paper titled “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks”, published in Aug 2019, was authored by Nils Reimers and Iryna Gurevych and presented in thearXiv repository. [19] The paper titled “Retrival Augmented Generation-Based Chatbot for Prospective and Current University Students”, published in June 2025, was authored by Luluk Setiawati Hartono, Esther Irawati Setiawan, Vrijraj Singh and presented in International Journal of Engineering, Science and Information Technology (IJESTY) Volume 5, No. 3 (2025) pp.268-277 elSSN: 2775-2674. [20] The paper titled “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, published in April 2021, was authored by Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rocktaschel, Sebastian Riedel, Douwe Kiela and presented in arXiv repository. [21] The paper tiltled “Enhancing LLM Factual Accuracy with RAG to Counter Hallucinations: A Case Study on Domain-Specific Queries in Private Knowledge-Bases, published in March 2024, was authored by Jiarui Li, Ye Yuan, Zehua Zhang and presented in arXiv repository

Copyright

Copyright © 2025 Amiya P Bovas, Mersa Joy, Nikhitha Catharin Santo, Shreya Ravindra Borde, Vibhav V Deshpande. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.