Asclepius AI: A Thorough Multi-modal Machine Learning Framework for Disease Prediction from Symptoms and Images

Abstract

The increasing digitization of healthcare hasled to an urgent need for scalable, accurate, and reliable com-putational tools that can meaningfully interpret heterogeneous patientdata.Inclinicalinformatics,traditionaldiagnosticmodels are usually built around a single modality of data, such as laboratory blood panels, isolated dermatological photographs, or individualpatient-reportedsymptoms.Thisinherentlyconstrains their clinical utility and generalizability to real-world situations. These unimodal systems are ill-suited to capture the complex cross-modal physiological realities underlying systemic diseases. To overcome these limitations, this paper introduces Asclepius AI: a comprehensive, multi-modal digital health platform that connects patient-reported symptom data with high-accuracy Artificial Intelligence (AI) based diagnostic reasoning. To tackle the intrinsic complexity and high dimensionality of the medical inputs,theproposedframeworkemploysathree-layerdiagnostic architecture. The first tier, designated the Basic Engine, applies an ensemble Random Forest classifier for rapid, interpretable screening of straight forward symptom presentations. Thesecond tier,theAdvancedEngine,leveragesacarefullyoptimizedArtifi-cialNeuralNetwork(ANN)capableofhigh-dimensionalsystemic disease prediction spanning 41 distinct disease categories. The thirdtier,theSkinEngine,employsafine-tunedEfficientNet-V2-S convolutional model for the classification of dermatological images. The validation accuracy of the Skin Engine reaches 96.25% on a heterogeneous dataset of 22,057 augmented derma-tological images and wide symptom vectors, achieving state-of-the-art results. In addition to its predictive engines, the platform is deployed as an integrated full-stack ecosystem built on Next.js and FastAPI, with a Large Language Model (LLM)-powered conversational agent for interactive health guidance and secure Electronic Health Record (EHR) management supported by PostgreSQL and Prisma ORM. Together, these results show that thesynergyofmulti-modaldeeplearningandpersistentconversa-tionalcontextcandramaticallyshortendiagnosticworkflowsand develop a transparent and clinically actionable decision support system.

Introduction

Artificial Intelligence (AI) and Machine Learning (ML) are transforming healthcare by enabling faster, more accurate, and data-driven disease diagnosis. However, many existing AI systems face challenges such as limited real-world performance, reliance on single data types, lack of transparency, and poor integration of diverse clinical information. To address these issues, the study introduces Asclepius AI, a multimodal clinical decision-support platform that combines symptom analysis, skin image classification, and conversational AI to assist in early disease screening rather than replace physicians.

The literature review highlights that diagnostic accuracy improves when AI models integrate multiple data types and use explainable AI techniques. Previous studies demonstrated the effectiveness of ensemble learning, artificial neural networks (ANNs), and EfficientNet architectures for disease prediction and medical image analysis, while emphasizing the importance of model interpretability for clinical trust.

Asclepius AI is designed as a secure, web-based platform consisting of four main components: a user-friendly frontend, a FastAPI-based diagnostic routing system, a Groq-powered conversational AI using LLaMA-3, and a secure PostgreSQL database with Cloudinary storage. The system supports patients by maintaining longitudinal health records, providing AI-assisted health guidance, and enabling secure sharing of diagnostic reports with physicians.

The platform uses three specialized machine learning engines:

• Basic Engine: A Random Forest classifier for patients reporting fewer than five symptoms.
• Advanced Engine: A regularized Artificial Neural Network (ANN) that predicts 41 systemic diseases from a 132-dimensional symptom vector.
• Skin Engine: An EfficientNet-V2-S deep learning model for classifying dermatological diseases from clinical images.

A multimodal routing algorithm automatically selects the appropriate prediction model based on whether the input consists of symptoms or images. Data preprocessing includes one-hot encoding of symptom data, image resizing and normalization, and extensive data augmentation to improve model robustness.

The implementation uses Python, TensorFlow, PyTorch, FastAPI, Next.js, PostgreSQL, and Cloudinary. Models were trained on Google Colab Pro using NVIDIA Tesla T4 GPUs and optimized for deployment through ONNX export and FP16 quantization, enabling efficient CPU-based inference without requiring dedicated GPUs.

Conclusion

Overall, Asclepius AI presents a scalable, interpretable, and multimodal AI-assisted healthcare platform that integrates advanced machine learning with secure clinical workflows to improve early disease screening, patient engagement, and clinical decision support.

References

[1] Mohamed, M. Abdelrehim, and R. Al-Barazie, “Context matters inmachine learning based disease prediction with insights from diverseclinical and symptom data,” Scientific Reports, vol. 15, p. 42669, 2025. [2] R. Zhang et al., “Multimodal artificial intelligence in medicine: a task-oriented framework for clinical translation,” Frontiers in Medicine, vol.12, p. 1736272, Jan. 2026. [3] L.Meincke,C.Terwiesch,andA.Huchzermeier,“EvaluatingLLMsforDynamic,MultimodalClinicalDecision-Making,”MackInstituteforIn-novationManagement,TheWhartonSchool,UniversityofPennsylvania,Jan. 2026. [4] G. Mohan et al., “Disease Prediction Based on Symptoms UsingEnsemble and Hybrid Machine Learning Models,” Amrita School ofComputing, 2023. [5] M. Alruwaili and M. Mohamed, “An Integrated Deep Learning Modelwith EfficientNet and ResNet for Accurate Multi-Class Skin DiseaseClassification,” Diagnostics, vol. 15, no. 5, p. 551, Feb. 2025. [6] P. Meenalochini, “Explainable AI for Healthcare Diagnostics: A Multi-modal Approach Using Vision and Language Models,” Journal ofArtificial Intelligence and Cyber Security (JAICS), vol. 9, no. 1, 2025. [7] P. Guleria, P. N. Srinivasu, S. Ahmed, N. Almusallam, and F. K.Alarfaj, “XAI framework for cardiovascular disease prediction usingclassification techniques,” Electronics, vol. 11, no. 24, Art. no. 4086,2022. [8] M. Wojtara, E. Rana, T. Rahman, P. Khanna, and H. Singh, “Artificialintelligence in rare disease diagnosis and treatment,” Clin. Transl. Sci.,vol. 16, no. 11, pp. 2106–2111, 2023. [9] M. M. Ahsan and Z. Siddique, “Machine learning-based heart diseasediagnosis: A systematic literature review,” Artif. Intell. Med., vol. 128,Art. no. 102289, 2022. [10] N. G. Nia, E. Kaplanoglu, and A. Nasab, “Evaluation of artificialintelligence techniques in disease diagnosis and prediction,” Discov.Artif. Intell., vol. 3, no. 1, Art. no. 5, 2023. [11] M. M. Ahsan, S. A. Luna, and Z. Siddique, “Machine-learning-baseddisease diagnosis: A comprehensive review,” Healthcare (Basel), vol.10, no. 3, Art. no. 541, 2022. [12] G.Stiglic,P.Kocbek,N.Fijacko,M.Zitnik,K.Verbert,andL.Cilar, “Interpretability of machine learning-based prediction models inhealthcare,”WileyInterdiscip.Rev.DataMiningKnowl.Discovery,vol.10, no. 5, Art. no. e1379, 2020. [13] G.Litjensetal.,“Asurveyondeeplearninginmedicalimageanalysis,”Med. Image Anal., vol. 42, pp. 60–88, 2017. [14] T. Bhanuteja, K. V. N. Kumar, K. S. Poornachand, C. Ashish, and P.Anudeep, “Symptoms-based multiple disease prediction model usingmachine learning approach,” Int. J. Innov. Technol. Explor. Eng., 2021. [15] P. C. Austin, I. R. White, D. S. Lee, and S. van Buuren, “Missing datain clinical research: a tutorial on multiple imputation,” Can. J. Cardiol.,vol. 37, no. 9, pp. 1322–1331, 2021. [16] R. Alkhanbouli, H. M. A. Almadhaani, F. Alhosani, and M. C. E.Simsekler, “The role of explainable artificial intelligence in diseaseprediction:asystematicliteraturereviewandfutureresearchdirections,”BMC Med. Inform. Decis. Making, vol. 25, no. 1, Art. no.

Copyright

Copyright © 2026 Prafulla B. Bhand, Aniruddha N. Rane, Ayush K. Khairnar, Soham S. Phatak, Punam A. Shinde, Dr. Emmanuel M.. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.