Conversational LLM-Based Multi-Modal Product Research Assistant

Abstract

With the continuous growth of e-commerce plat- forms, users often face challenges in find ing relevant, unbiased, and reliable product information across multiple sources. Existing recommendation systems are limited by static algorithms that lack contextual understanding and transparency. To overcome these challenges, this project proposes a Conversational LLM- Based Multi-Modal Product Research Assistant — an intelligent AI system designed to enable natural, context-aware, and data- driven product exploration. The proposed system integrates a domain-adapted Large Language Model (LLM) with multi-modal capabilities to process both textual and visual product data. Item ploys a Retrieval-Augmented Generation (RAG) framework to access real-time information from verified e-commerce sources, ensuring accuracy and up-to-date recommendations. Addition- ally, it incorporates unbiased comparison and sustainability- aware reasoning aligned with UN Sustainable Development Goal (SDG) 12– Responsible Consumption and Production. The outcome is a scalable, web-based assistant that allows users to converse naturally, view relevant images and specifications, and make informed purchasing decisions. This project demonstrates how conversational AI and multi-modal learning can transform digital shopping into a transparent, personalized, and sustainable experience.

Introduction

This paper proposes a Conversational LLM-Based Multi-Modal Product Research Assistant, an AI-powered system designed to improve product research and comparison in e-commerce. With the rapid growth of online shopping, users face information overload due to large product catalogs, multiple vendors, and inconsistent data sources. Traditional recommendation systems, based on collaborative filtering and content-based methods, often fail to understand user intent, adapt to changing preferences, or provide transparent cross-platform comparisons.

Background

Recent advances in Large Language Models (LLMs) such as LLaMA 3, GPT-4, and Mistral have enabled more intelligent and conversational interactions. These models can understand natural language queries, maintain context across multiple interactions, and reason over information from diverse sources. Combined with multi-modal learning, they can process both text and images, allowing richer product exploration.

Proposed Solution

The proposed system uses:

• Retrieval-Augmented Generation (RAG) to obtain real-time information from verified e-commerce sources.
• Multi-modal learning to analyze both product images and textual descriptions.
• Conversational AI for context-aware, multi-turn interactions.
• Sustainability indicators such as recyclability and environmental impact, supporting the goals of sustainable consumption (UN SDG 12).

Problem Statement

Current e-commerce recommendation systems:

• Depend on platform-specific data.
• Produce biased or repetitive recommendations.
• Lack contextual understanding of user needs.
• Require users to manually verify product specifications, prices, and authenticity.
• Do not provide comprehensive cross-platform comparisons.

Motivation

The work is motivated by the growing demand for:

• Transparent and trustworthy product research.
• Better-informed purchasing decisions.
• Sustainable and ethical consumption.
• Reduced user effort through intelligent automation.

Objectives

The research aims to:

1. Develop a conversational, multi-modal product assistant.
2. Integrate domain-adapted LLMs for contextual interactions.
3. Combine textual specifications and product images.
4. Provide unbiased product comparisons using RAG.
5. Deliver personalized recommendations while preserving privacy through session-based context management.
6. Create a scalable web-based platform for real-world deployment.

Scope

The system:

• Aggregates data from multiple e-commerce platforms.
• Analyzes both visual and textual product information.
• Generates structured, fact-based insights.
• Supports future expansion through multilingual capabilities, cloud deployment, and additional product categories.
• Emphasizes transparency, scalability, and sustainability.

Literature Survey Findings

The review of prior research shows progress in:

• Web scraping and data extraction.
• Product matching and comparison.
• Recommendation systems.
• Multi-modal learning.
• LLM-based reasoning and evaluation.

These studies have improved retrieval accuracy, personalization, explainability, product matching, and recommendation quality.

Research Gaps Identified

1. Data Acquisition Challenges

• Difficulty handling dynamic JavaScript-based websites.
• Dependence on fragile, manually crafted extraction rules.
• Scalability and maintenance issues.

2. Recommendation System Limitations

• Insufficient contextual understanding.
• Weak integration of text and image information.
• Limited connection between recommendation engines and conversational reasoning.

3. Lack of Sustainability Metrics

• Most systems ignore environmental impact, recyclability, and ethical manufacturing factors when generating recommendations.

Conclusion

This paper presented a Conversational LLM-Based Multi- Modal Product Research Assistant aimed at addressing some of the limitations of traditional e-commerce recommendation systems. By integrating large language models with retrieval- augmented generation and multi-modal processing, the pro- posed system enables more context-aware and user-friendly product exploration. The prototype implementation demonstrates that the system is capable of handling natural language queries, retrieving relevant product information from multiple sources, and gen- erating structured responses to assist users in decision-making. The inclusion of multi-modal components allows the system to incorporate both textual and visual information, improving the overall quality of product analysis. Additionally, the use of retrieval-based mechanisms helps in grounding responses and reducing inaccurate or unsupported outputs. The integration of basic sustainability indicators aligned with UN SDG 12 highlights the potential of incorporating responsible consumption aspects into recommendation systems. While the current implementation is limited to a prototype environment, it shows promising results in improving trans- parency and usability in product research. However, further improvements are required to enhance scalability, robustness, and real-world performance. Future work may focus on expanding the dataset, improving multi- modal reasoning capabilities, and deploying the system in a fully interactive and production-ready environment. Overall, this work demonstrates the feasibility of combining con- versational AI and retrieval-based approaches for building intelligent product research assistants.

References

[1] J.-C. Bricongne, B. Meunier, and S. Pouget, “Web-scraping housing prices in real-time: The Covid-19 crisis in the UK,” Journal of Housing Economics, 2023. [2] V. Pichiyan, S. Muthulingam, G. Sathar et al., “Web Scraping using Natural Language Processing: Exploiting Unstructured Text for Data Ex- traction and Analysis,” Procedia Computer Science, vol. 230, pp. 193– 202, 2023. [3] O. Jorge, A. Pons, J. Rius et al., “Increasing online shop revenues with web scraping: A case study for the wine sector,” British Food Journal, vol. 122, no. 11, pp. 3383–3401, 2020. [4] M. D’Souza, S. Desai, D. Agrawal, F. Joshi et al., “Web scraping based product comparison model for e-commerce website,” Journal of Emerging Technologies and Innovative Research, vol. 11, no. 4, 2024. [5] P. Bandi, “Advanced Google Scholar Scraper: A content-based filtering approach for literature recommendation using BERT,” M.Sc. Research Project, National College of Ireland, 2023. [6] P. Galvez-Hernandez, A. Gonzalez-Viana, L. Gonzalez-de Paz et al., “Generating contextual variables from web-based data for health re- search,” JMIR Public Health Surveillance, vol. 10, 2024. [7] J. Y. Guyt, H. Datta, and J. Boegershausen, “Unlocking the potential of web data for retailing research,” Journal of Retailing, vol. 100, pp. 130– 147, 2024. [8] J.-K. Kim et al., “A Knowledge Graph and Large Language Model- Based Hybrid Recommender System,” in Advances in Intelligent Systems and Computing, Springer, 2023. [9] Y. Wang, Z. Chu, X. Ouyang et al., “Enhancing recommender sys- tems with large language model reasoning graphs,” arXiv preprint arXiv:2308.10835, 2023. [10] X. Ren, W. Wei, L. Xia et al., “Representation learning with large lan- guage models for recommendation,” arXiv preprint arXiv:2310.15950, 2023. [11] A. Nigam, P. Jain, and A. Gupta, “Semantic product search for matching structured product catalogs in e-commerce,” arXiv preprint arXiv:2008.08180, 2020. [12] Y. Zhang, X. Liu, and C. Yu, “Learning variant product relationship and variation attributes from e-commerce website structures,” in Proc. ACM Web Conference (WWW), 2022. [13] H. Chen, M. Zhang, and D. Song, “Effective product schema matching and duplicate detection with large language models,” arXiv preprint arXiv:2104.09576, 2021. [14] X. Wang, L. Zhao, and R. Li, “AE-smnsMLC: Multi-label classification with semantic matching and negative label sampling for product attribute value extraction,” arXiv preprint arXiv:2310.07137, 2023. [15] A. Brinkmann, R. Shraga, and C. Bizer, “ExtractGPT: Exploring the potential of large language models for product attribute value extraction,” arXiv preprint arXiv:2310.12537, 2024. [16] C. Fang, X. Li, Z. Fan et al., “LLM-Ensemble: Optimal large language model ensemble method for product attribute value extraction,” in Proc. 47th Int. ACM SIGIR Conference, 2024. [17] Z. He, M. Huang, and Q. Lin, “LaTeX-Numeric: Language-agnostic text attribute extraction for e-commerce numeric attributes,” arXiv preprint arXiv:2403.00863, 2024. [18] M. S. Baysan, S. Uysal, I. Islek et al., “LLM-as-a-Judge: Automated evaluation of search query parsing using large language models,” Fron- tiers in Big Data, vol. 8, 2025. [19] E. Jeong, X. Li, A. Kwon et al., “A multimodal recommender system using deep learning techniques combining review texts and images,” Applied Sciences, vol. 14, no. 20, 2024. [20] C. Li, Y. Deng, H. Hu et al., “CARE: Contextual adaptation of recommenders for LLM-based conversational recommendation,” arXiv preprint, 2025.

Copyright

Copyright © 2026 Aditya Bhosale, Sandeep Warhade, Nihal Barkade, Akshay Choudhari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.