Dermavision :Transforming Skincare, One Recommendation at a Time

Abstract

DermavisionisanAI-drivenskincarerecommenda- tion system that integrates deep learning, computer vision, and augmentedreality(AR)toprovidepreciseandreal-timeskincare solutions. Traditional skincare recommendation systems often rely on sentiment analysis, quizzes, or static image processing, leading to subjective and less accurate results. Dermavision enhancespersonalizationthroughmachinelearningmodels,clus- tering in YCbCr and HSV color spaces for accurate skin classifi- cation, and AI-driven allergy detection. The system incorporates a virtual ARbased skincare application, an AI chatbot for real- time skincare guidance, and a community-driven platform for userengagement.Thispaperdetailsthemethodology,implemen- tation, and technological advancements that make Dermavisiona leading innovation in AI-based skincare recommendations. Unlike existing systems that rely on sentiment analysis or static image evaluations, Dermavision leverages deep learning-based skin classification, real-time AI-driven allergy detection, and AR visualization for scientifically backed recommendations.

Introduction

1. Problem in Existing Systems

• Many AI-based skincare tools rely on user input, static images, or subjective quizzes, leading to inaccurate and generalized recommendations.

• Key limitations:

  • Lack of real-time skin analysis

  • No allergy detection

  • Inadequate skin disease classification

  • Weak AR integration for visualization

2. Dermavision: A New Approach

Dermavision overcomes these gaps using:

• Real-time image analysis via CNNs (Convolutional Neural Networks)

• Fitzpatrick scale for skin tone classification

• AR technology for visualizing skincare outcomes

• Ingredient safety assessment to avoid allergens

• Community features for user interaction and advice-sharing

3. Literature Survey Insights

• Prior systems used:

  • OpenCV and CNNs for skin analysis

  • Sentiment-based recommendations (subjective)

  • AR for virtual try-ons, but lacking clinical accuracy

• Dermavision improves by:

  • Using EfficientNetB0 for skin classification

  • K-means clustering for enhanced skin segmentation

  • Integrating image-based allergy detection and chatbot assistance

  • Offering scientifically backed recommendations

4. Key Technologies Used

• Face detection & preprocessing with face-api.js and OpenCV

• Skin segmentation using Otsu’s Thresholding, HSV & YCbCr filtering

• K-Means Clustering for isolating skin regions

• CNN (EfficientNetB0) to classify skin types (dry, oily, etc.) and acne severity

• Fitzpatrick scale mapping for accurate skin tone detection

• KNN Algorithm for final skin classification

• Cosine similarity to match user skin profiles with suitable products

5. Implementation Details

• Frontend built in React.js with:

  • AR visualization using TensorFlow.js FaceMesh

  • Chat support and personalized dashboard

• Backend supports:

  • Real-time analysis

  • Product recommendation engine

  • Dataset: Kaggle Acne Grading (enhanced via data augmentation)

6. Unique Features

• Combines AI, AR, clustering, and dermatology into one platform

• Minimizes user input dependency

• Offers personalized, explainable, and data-driven skincare guidance

• Plans for future: Use of clinically validated datasets and real-time feedback loops

Conclusion

The development of Dermavision marks a significant ad- vancement in AI-driven skincare, offering a personalized and data-driven approach through computer vision, deep learning, and clustering techniques. Key features such as real-time im- ageprocessing,augmentedreality-basedproductvisualization, an AI-powered chatbot, and a community-driven platform enhance user engagement and accessibility. While the sys-tem improves recommendation accuracy and user experience, ongoing refinements are essential. Over the next 6 to 12 months, efforts will focus on optimizing machine learning models, expanding dermatological datasets, refining chatbot responsiveness, and enhancing long-term skin health predic- tions. Future updates will explore wearable device integration for real-time skin monitoring and proactive skincare insights. With continuous innovation, Dermavision has the potential to set a new benchmark in AI-powered skincare, bridging thegap between artificial intelligence and personalized beauty solutions.

References

[1] L. M. I. T. Hemantha, T. M. E. Gayathri, and N. N. M. De Silva,”Personalized Smart Skincare Product Recommendation System,” SriLanka Institute of Information Technology, Malabe, Sri Lanka, 2023. [2] H.-T. Chan, T.-Y. Lin, S.-C. Deng, C.-H. Hsia, and C.-F. Lai, ”SmartFacial Skincare Products Using Computer Vision Technologies,” Inter-nationalJournalofDermatologicalResearch,vol.45,no.2,pp.150-165,2022. [3] M.Jayaram,S.A.Reddy,B.Praneetha,M.Pujitha,C.S.Chandra,and K. B. Prakash, ”Smart Cosmetics Recommendation System UsingAI,” International Journal of Advanced Computing, vol. 39, no. 4, pp.98-112, 2023. [4] Z. Wu et al., ”Studies on Different CNN Algorithms for Face SkinDisease Classification Based on Clinical Images,” IEEE Access, vol. 7, [5] pp.66505-66511,2019,doi:10.1109/ACCESS.2019.2918221. [6] B. Lokesh, A. Devarakonda, G. Srinivas, and N. K. Naik, ”IntelligentFacial Skin Care Recommendation System,” Machine Learning forHealthcare Applications, vol. 17, no. 3, pp. 200-215, 2023. [7] K.H.NagarajaiahandM.Hanakere,”ArtificialIntelligence-BasedSmartCosmetics Suggestion System Based on Skin Condition,” Journal of AIand Beauty Tech, vol. 12, no. 1, pp. 40-58, 2023. [8] S. Jadhav, D. Memane, K. Supekar, S. Shinde, and T. Jadhav, ”Person-alized Skin Care Recommendation Using Machine Learning,” CieˆnciaEngenharia - Science Engineering Journal, vol. 11, no. 1, pp. 112-128,2023. [9] P. N. Maduranga and D. Nandasena, ”Mobile-Based Skin DiseaseDiagnosis System Using Convolutional Neural Networks (CNN),” In-ternational Journal of Image, Graphics and Signal Processing, vol. 14,no. 3, pp. 47-57, 2022, doi: 10.5815/ijigsp.2022.03.05. [10] C. Isakowitsch, ”How Augmented Reality Beauty Filters Can AffectSelf-Perception,” in Artificial Intelligence and Cognitive Science, L.LongoandR.O’Reilly,Eds.Cham:Springer,2023,vol.1662,pp.78-92. [11] J. Preece and D. Maloney-Krichmar, ”Online Communities: Design,Theory, and Practice,” Journal of Computer-Mediated Communication,vol. 10, no. 4, Jul. 2005, doi: 10.1111/j.1083-6101.2005.tb00264.x. [12] A. X. Du, S. Emam, and R. Gniadecki, ”Review of Machine Learningin Predicting Dermatological Outcomes,” Frontiers in Medicine, vol. 7,no. 266, Jun. 2020, doi: 10.3389/fmed.2020.00266. [13] R. Rastogi, M. Shahjahan, P. Yadav, and M. Gupta, ”Deep Learning forFacial Skin Issues Detection: A Study for Global Care With Healthcare5.0,” in Proceedings of the IEEE Conference on Healthcare AI, NewYork, USA, 2023, pp. 89-105. [14] G. S. Tompunuh and A. P. Wibowo, ”Skincare and Make-Up Intro-duction: Why Not Use Augmented Reality?” International Journal ofComputer Applications, vol. 185, no. 44, Nov. 2023. [15] N. A. Windasari, N. Shafira, and H. B. Santoso, ”Augmented RealityExperiential Marketing in Beauty Products: Does it Differ from OtherService Touchpoints?” Jurnal Sistem Informasi (Journal of InformationSystems), vol. 18, no. 2, pp. 50-67, 2022. [16] B. Handelshøyskolen, ”The Effect of Artificial Intelligence and ProductRecommendation on Purchase Intention: A Study of the CosmeticsField,” WISEflow Europe/Oslo, Jun. 2023. [17] N. Mangtani, N. Bajpai, S. Sahasrabudhe, and D. Wasule, ”Importanceof Artificial Intelligence and Augmented Reality in the Cosmetic andBeauty Industry Post COVID-19,” World Journal of PharmaceuticalResearch, vol. 14, no. 3, pp. 115-130, 2023. [18] A. Bar?s¸, ”A New Business Marketing Tool: Chatbot,” GSI JournalsSeries B: Advancements in Business and Economics, vol. 3, no. 1, pp.31-46, 2020. [19] J. Smith et al., ”Artificial Intelligence in Dermatology: Deep Learning-BasedSkinAnalysisandReal-TimeMonitoring,”ComputersinBiologyand Medicine, vol. 170, 2024. [20] R. Lee et al., ”Deep Learning and Clustering Techniques for Dermato-logical Image Segmentation,” Intelligent Medicine Review, vol. 58, pp.205-220, 2022. [21] S. Patel et al., ”AI-Powered Ingredient Analysis and Allergy Detectionin Dermatology,” Journal of Skin ScienceAI, vol. 37, pp. 75-89, 2022. [22] R. Lathiya, ”Acne Grading Classification Dataset,” Kaggle, 2022.[Online]. Available: https://www.kaggle.com/rutviklathiyateksun/acne-grading-classification-dataset

Copyright

Copyright © 2025 Bhakthi Shetty, Nikita Shetty, Saloni Suvarna, Prof. Rajesh Kolte. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.