Beyond Black Box: How LLMs Work with RAG to Deliver Personalized Financial Advisory

Abstract

Financial literacy among young people, combined with the expense and unavailability of professional financial advisory services, leaves a huge gap in personalized financial advice, especially in developing countries like India. The current state-of-the-art Large Language Model (LLM)-based systems, despite their interactive and conversational capabilities, have some serious drawbacks such as hallucination, generation of incorrect information, and data security issues, making them unsuitable for financial advisory tasks. This paper presents a Personalized Financial Advisory Chatbot (PFAC) system that combines LLMs with a Retrieval-Augmented Generation (RAG) model to provide accurate, reliable, and accessible financial advice using natural language conversations. The RAG model uses the retrieval of verified information from carefully selected financial knowledge sources to effectively mitigate hallucinations and improve the reliability of responses. The proposed system provides a scalable, affordable, and democratized approach to empower people to make informed financial choices.

Introduction

The text discusses the development of a personalized financial advisory chatbot using Retrieval-Augmented Generation (RAG) to provide accurate, trustworthy, and context-aware financial advice, particularly for the Indian financial ecosystem. Traditional financial guidance has often been inaccessible to many people, while general AI assistants such as OpenAI’s ChatGPT and Microsoft Copilot may produce generic or outdated financial responses due to limited domain-specific knowledge and hallucination issues.

To solve this problem, the proposed framework uses Retrieval-Augmented Generation (RAG), which combines information retrieval with large language models (LLMs). Instead of relying only on pre-trained model knowledge, the system retrieves relevant financial documents during inference and uses them as context to generate accurate and grounded responses. The framework integrates domain-specific financial knowledge bases, semantic embeddings, vector similarity search, prompt engineering, and response generation techniques to improve reliability and reduce hallucinations.

The literature survey reviews previous studies on LLMs, vector databases, LangChain, embeddings, and RAG-based chatbots. Existing systems often lack domain-specific financial knowledge, hallucination control, personalization, explainability, and privacy-preserving local deployment. Many rely on cloud infrastructure, which raises concerns regarding cost and data privacy. The proposed system, called PFAC, aims to fill these gaps by offering a complete, locally deployable, privacy-focused, and hallucination-controlled financial advisory chatbot.

The paper also explains key concepts used in the framework:

• RAG improves factual accuracy by retrieving external knowledge during response generation.
• Large Language Models (LLMs) are powerful text-generation systems but may produce hallucinated information.
• Embeddings such as TF-IDF and Word2Vec convert text into semantic vector representations for similarity matching.
• Vector stores enable efficient storage and retrieval of embedded financial documents.
• Prompt engineering uses carefully designed prompts to guide the model toward accurate, domain-specific responses.
• Hallucination refers to the generation of believable but incorrect information by LLMs.

References

[1] Bind AI, “Comparing ChatGPT, Bard, LLaMA, and Custom LLM Applications for Financial Information from SEC Data,” Medium, 2023. [2] N. Yadav, “Building an Intelligent Chatbot with LangChain and Vector Databases,” Medium, 2023. [3] M. Younes, “Creating a Chatbot with LLM, LangChain, and Vector Databases,” Medium, 2023. [4] Qwak, “Utilizing LLMs with Embedding Stores to Improve Context,” Qwak Blog, 2023. [5] R. Kerr, “Creation of a Custom Generative AI Chatbot Using Grounded Data,” robkerr.com, 2023. [6] Beyond The Board, “LLM-Based AI Chatbots: Architecture, Applica- tions, and Future Directions,” Medium, 2023. [7] Google Cloud, “How to Build a Financial Advisory Bot Using RAG with GenAI,” Google Cloud Blog, 2023. [8] R. Sharma and P. Mehta, “Personalized Finance Chatbot Powered by RAG and Generative AI for Smart Wealth Management,” International Journal of Engineering Research and Technology (IJERT), vol. 14, no. 3, 2025. [9] A. Kumar and S. Verma, “Role-Based Chatbot Creation Using Artificial Intelligence,” International Journal of Research Publication and Reviews (IJRPR), vol. 6, no. 6, 2025. [10] Bright AI, “Choosing the Right Embedding Models for RAG in Gener- ative AI Applications,” Medium, 2023. [11] CFA Institute Research and Policy Center, “RAG for Finance: Automat- ing Document Analysis with LLMs,” Automation Ahead Content Series, CFA Institute, 2024. [12] Author(s), “Chapter Title,” in Book/Proceedings Title, Springer, 2024, pp. XX–XX, doi: 10.1007/978-3-032-11976-6 20. [13] Author(s), “Title of the Paper,” arXiv preprint arXiv:2504.14493, 2025. [14] M. Luckianto and A. A. S. Gunawan, “MARAG-Fin: An Intelligent Multi-agent RAG-LLM Architecture Integrating Financial News Senti- ment and Time Series Data for Data-driven Trading Decision-making,” International Journal of Intelligent Engineering and Systems, vol. 19, no. 2, pp. 738–750, 2026, doi: 10.22266/ijies2026.0228.46. [15] Lumenova AI, “AI in Finance: The Promise and Risks of RAG,” Lumenova AI Blog, 2024.

Copyright

Copyright © 2026 Dr. A. Kannammal, Vaisaali Priyankaa R, Dhrisheta M S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.