Mental Health Monitoring System

Abstract

Mental health challenges are increasing globally, while access to timely and affordable support remains limited. This paper presents an AI-based Mental Health Monitoring System that provides real-time emotional analysis and adaptive support using a multimodal approach. The system integrates facial emotion recognition using Convolutional Neural Networks (CNN), voice tone analysis using Web Audio APIs, and text sentiment analysis through Natural Language Processing techniques. The system has evolved from an initial prototype into a full-stack platform incorporating secure user authentication, session lifecycle management, journaling, and emotion analytics. It follows a client-server architecture where backend services handle processing and storage using a PostgreSQL database. The platform is accessible via web and mobile browsers through Progressive Web App (PWA) support. The system demonstrates real-time performance with low latency and provides context-aware responses through an AI-driven chatbot. This solution serves as an assistive tool for emotional awareness and early intervention.

Introduction

The text presents an AI-based multimodal mental health monitoring system designed to detect and support users’ emotional well-being in real time.

Mental health issues such as anxiety, depression, and stress are widespread, but many people avoid seeking help due to stigma, cost, and limited access to professionals. To address this, the proposed system uses artificial intelligence and affective computing to automatically detect emotions and provide timely, supportive feedback.

The system combines three modalities for emotion recognition:

• Facial expressions (via webcam)
• Voice tone analysis (via microphone)
• Text sentiment analysis (user input)

These inputs are processed through a full-stack architecture where a backend system manages authentication, session tracking, and data storage, while a frontend interface allows real-time interaction. A conversational AI chatbot responds dynamically based on detected emotions, offering empathetic and context-aware support. The system also includes session tracking, journaling, and analytics dashboards to help users understand emotional patterns over time.

The literature review highlights recent advances in deep learning, transformer models, EEG-based emotion recognition, and chatbot-based therapy systems, noting a trend toward multimodal approaches. However, it also identifies challenges such as computational complexity, privacy concerns, and limited real-world validation.

Experimental results show that the system performs efficiently in real-time, with low latency and improved accuracy due to multimodal fusion compared to single-method approaches. Users benefit from adaptive chatbot responses and emotional analytics that enhance self-awareness.

In conclusion, the system demonstrates how AI and multimodal emotion detection can support mental health monitoring, making emotional support more accessible, scalable, and interactive. Future improvements include integration of biometric signals, multilingual support, wearable device integration, and deployment as a full mobile application.

Conclusion

This paper presents a multimodal AI-based mental health monitoring system that integrates facial emotion recognition, voice tone analysis, and text sentiment analysis to provide real-time emotional assessment and adaptive support. Unlike traditional approaches that rely on a single input modality, the proposed system combines multiple data sources to improve the accuracy and reliability of emotion detection. The system has evolved from an initial prototype into a full-stack application, incorporating user authentication, session lifecycle management, journaling, and emotion analytics. The integration of a conversational AI chatbot enables context-aware and empathetic interactions, enhancing user engagement and overall experience. Real-time processing capabilities and low latency ensure smooth interaction, while analytics features allow users to track emotional trends over time. The system demonstrates how AI can be effectively applied to create accessible, scalable, and user-friendly mental health support solutions. Overall, the proposed system highlights the potential of integrating multimodal emotion detection with conversational AI to support early mental health awareness and intervention.

References

[1] J. Zhang, et al., “EEG Multimodal Fusion Review,” IEEE Access, 2025. [2] X. Zhu, et al., “SOGPCN: Graph-Based Pseudo-3D CNN for EEG Emotion Recognition,” Sensors, 2024. [3] Y. Pan, et al., “DuA: Dual Attentive Transformer for Long-Term EEG Analysis,” arXiv preprint, 2024. [4] G. Xiao, et al., “Four-Dimensional Attention Neural Network for Emotion Recognition,” Sensors, 2024. [5] Y. Chu, et al., “MMESC: Multimodal Emotion Support Conversations,” 2023. [6] R. Yuvaraj, et al., “3D Convolutional Neural Networks for SEED and SEED-IV Emotion Datasets,” Brain Sciences, 2023. [7] S. Sadeh-Sharvit, et al., “AI-Based Behavioral Therapy Platform for Mental Health,” JMIR, 2023. [8] D. Danieli, et al., “Therapist-Informed Participatory Design of AI Mental Health Agents,” 2021. [9] A. Khanzada, et al., “Facial Emotion Recognition Using CNN on FER-2013 Dataset,” in Proc. IEEE Conf. Artificial Intelligence, 2020.

Copyright

Copyright © 2026 Miss. Shreya Bhise, Miss. Shreya Rane, Miss. Meghana Shivaprasad, Prof. Shraddha Katkar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.