AI-Powered Visual Search and Virtual Try-On for E-Commerce

Abstract

Thee-commercefashionindustryfacessignificantchallengesinenhancingproductdiscoveryandimprovingtheonline shoppingexperience.Customersoftenstruggletofindvisuallysimilarclothingitemsandaccuratelyvisualizehowgarmentswill fitandlookwhenworn.Theselimitationsleadtohigherreturnratesandreducedcustomersatisfaction.Toaddresstheseissues, this research proposes an intelligent e-commerce framework integrating Visual Search and Virtual Try-On functionalities. The VisualSearchmoduleenablesuserstouploadanimage,whichisprocessedusingaResNet-basedfeatureextractor.AK-Nearest Neighbors (KNN) algorithm then retrieves the top five visually similar products from a precomputed database, streamlining product discovery. The Virtual Try-On module utilizes pose detection and offset max pooling to accurately align and overlay clothing items on the user’s video input, offering a realistic preview of the garment on the user. This system enhances the online shopping experience by mitigating common challenges such as difficulty in finding similar products and uncertainty in garment fit and appearance. Future enhancements may include expanding the range of product categories, improving pose detection and clothing alignment, incorporating size recommendation models, and integrating augmented reality (AR) for an immersive shopping experience. By addressing these critical pain points, this research aims to revolutionize online fashion retail, improve customer engagement, and reduce return rates.

Introduction

The text discusses the transformation of retail through e-commerce, highlighting current challenges like inefficient text-based search and the inability to virtually try on clothes, which affects user satisfaction and leads to high return rates. To address these issues, the research proposes an AI-powered e-commerce platform that integrates Visual Search (using ResNet and K-Nearest Neighbors) to find products via images, and Virtual Try-On (using pose detection and image overlay) to let users realistically preview clothing on themselves.

The study leverages deep learning architectures like ResNet for feature extraction, and pose detection models for accurate garment fitting, using datasets such as DeepFashion and VITON. The system architecture supports smooth image upload, feature extraction, similarity matching, pose estimation, and real-time rendering to enhance product discovery and consumer confidence.

The research also emphasizes environmental benefits by potentially reducing return-related logistics and associated carbon emissions. Experiments show that the system provides accurate, efficient, and user-friendly solutions, with future improvements suggested like size recommendation and augmented reality integration for broader e-commerce applications.

Conclusion

The integration of visual search and virtual try-on technologies in e-commerce represents a significant advancement in enhancing user experience and streamlining product discovery. Traditional e-commerce platforms often rely on text-based search methods, which can be inefficient and restrictive. Additionally, the inability to visualize products accurately before purchase remains a key challenge for online shoppers.TheproposedAI-poweredframeworkaddressestheselimitationsbyleveragingdeeplearningandcomputervisiontechniquestooffer an intuitive, interactive, and personalized shopping experience.Throughextensiveexperimentationandimplementation,wedemonstratedtheeffectivenessofoursystem.Thevisualsearchmodule,powered by ResNet for feature extraction and K-Nearest Neighbors (KNN) for product retrieval, enables users to find products effortlessly, reducing dependency on text-based queries. The virtual try-on feature enhances the shopping experience by allowing users to visualize apparel in real time using pose detection and offset max pooling techniques. This not only boosts customer confidence but also contributes to lower return rates, addressing a major operational challenge in e-commerce.Beyond improving product discovery, the system fosters inclusivity by providing an intuitive user interface accessible to individuals from diverse backgrounds, including those facing language or literacy barriers. By focusing on seamless integration with existing e-commerce platforms, the proposed framework offers a scalable solution that enhances both customer satisfaction and business efficiency.

References

[1] T.Islam,A.Miron,X.Liu,andY.Li,“StyleVTON:AMulti-poseVirtualTry-On with Identityand ClothingDetailPreservation,” Neurocomputing,2024. [2] J.Gou,S.Sun,J.Zhang, J.Si,C. Qian,andL.Zhang,“TamingthePowerofDiffusionModelsforHigh-QualityVirtualTry-Onwith AppearanceFlow,”inProc.31stACMInt.Conf.Multimedia(MM’23),2023,pp.7599–7607. [3] C.Mu,J.Zhao,G.Yang,J.Zhang,andZ.Yan,“TowardsPracticalVisualSearchEngineWithinElasticsearch,”inProc.ACMSIGIR Workshop on eCommerce (SIGIR 2018 eCom), ACM, New York, NY, USA, 2018, 8 pp. [4] R.Yu,X.Wang,and X.Xie,“VTNFP:An ImageBasedVirtualTry-OnNetworkwith BodyandClothingFeaturePreservation,”in Proc.ZIEEE/CVFInt. Conf.Comput.Vis.,2019,pp.10511–10520. [5] L.Zhu,D.Yang,T. Zhu,F.Reda,W.Chan,C.Saharia,M.Norouzi,and I.KemelmacherShlizerman,“TryOnDiffusion:ATaleofTwo UNets,”inProc.IEEEConf.Comput.Vis.Pattern Recog.(CVPR),2023,pp.4606–4615 [6] S.Choi,S.Park,M.Lee,andJ.Choo,“VITONHD:High-ResolutionVirtualTry-OnviaMisalignment-AwareNormalization,”in Proc. IEEE/CVFConf.Comput. Vis.Pattern Recog.,2021,pp. 14131–14140. [7] B.Fele,A.Lampe,P.Peer,and V.Struc,“CVTON:Context-DrivenImageBased VirtualTry-OnNetwork,”inProc.IEEE/CVFWinter Conf.Appl.Comput. Vis.(WACV),2022. [8] F.Zhao,Z.Xie, M.Kampffmeyer,H. Dong,S.Han, T.Zheng,T. Zhang,X.Liang,“M3D-VTON: AMonocular-to-3DVirtual Try-On Network,”inProc.IEEE/CVFInt.Conf. Comput.Vis.(ICCV),2021,pp. 13239–132494 [9] Z. Xie, Z. Huang, X. Dong, F. Zhao, H. Dong, X. Zhang, F. Zhu, and X. Liang, “GP-VTON: Towards General Purpose Virtual Try-On viaCollaborativeLocalFlowGlobal-ParsingLearning,”inProc.IEEEConf.Comput.Vis.PatternRecog.(CVPR),2023,pp.23550–23559 [10] L.Wang,X.Qian,X.Zhang,and X.Hou,“Sketch-Based ImageRetrievalwithMultiClusteringRe-Ranking,”IEEETrans.Circ.Syst. Vid.Technol., vol.30,no.12,pp.4929–4943, Dec. 2020. [11] K.Tang,X.Chen,andP.Song,“3DObjectRecognitioninClutteredSceneswith RobustShapeDescriptionand Correspondence Selection,”IEEETrans.,2017. [12] S.Ren,K.He,R.B.Girshick,X.Zhang,and J.Sun,“ObjectDetection Netw

Copyright

Copyright © 2025 Divya Muppalla, Pranjali , Rohit Mishra, Sakshee B Agarwal, Aparna M. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.