AI for Mental Health: Audio Sentiment Analysis

Abstract

Depression is becoming a major mental health issue, and many people don’t realize they have it because the symptoms can be hard to notice. The way a person talks can actually reveal a lot about how they’re feeling emotionally and mentally. This system is designed to detect depression by analysing both how someone speaks (like their tone, pitch, and rhythm) and what they say (their sentiment or emotions in words). The process starts with speech-to-text (STT) technology, which turns spoken words into written text. Then, Natural Language Processing (NLP) methods examine the text to understand its emotional tone. At the same time, advanced audio processing looks at how the speech sounds by extracting features like Mel-frequency cepstral coefficients (MFCCs), which help capture the characteristics of the voice. These different pieces of information both the way someone speaks and the meaning of their words are combined and analyzed using an advanced deep learning model called LSTM. This model has been trained on a well-balanced dataset to ensure accurate results. By studying patterns in both speech and text over time, the system can effectively detect signs of depression.

Introduction

This paper presents an AI-based depression detection system that combines audio features and linguistic analysis to improve the accuracy of identifying depression. Depression is a serious mental health condition characterized by persistent sadness, loss of interest, fatigue, and difficulty concentrating. Early detection is essential for timely intervention and improved mental well-being.

Existing depression detection systems mainly rely on speech characteristics such as tone, pitch, and rhythm, while ignoring the actual words and emotions expressed in speech. They also face challenges including dependence on a single data source, poor performance on imbalanced datasets, sensitivity to noise and speech variations, and limited real-time applicability.

To overcome these limitations, the proposed system integrates audio and textual information. Speech recordings are first preprocessed, and audio features such as Mel-Frequency Cepstral Coefficients (MFCCs), pitch, tone, and rhythm are extracted. Simultaneously, speech is converted into text using Speech-to-Text (Whisper) technology, and Natural Language Processing (NLP) is applied to perform sentiment analysis. These combined features are then processed using a Long Short-Term Memory (LSTM) deep learning model, which effectively captures sequential speech patterns for depression classification.

The system is trained using the DAIC-WOZ dataset under a supervised learning framework. To address class imbalance, the SMOTE technique is employed, ensuring that the model learns equally from depressed and non-depressed samples. The model is optimized using the Binary Cross-Entropy loss function and the Adam optimizer.

The proposed methodology follows a structured workflow consisting of speech preprocessing, feature extraction, speech-to-text conversion, sentiment analysis, feature fusion, dataset balancing, LSTM-based classification, and performance evaluation. The system's effectiveness is assessed using Accuracy, F1-Score, Mean Absolute Error (MAE), and Root Mean Square Error (RMSE).

Conclusion

IdeaCrafter successfully addresses the challenges faced by aspiring entrepreneurs by providing an intelligent and scalable mentorship platform. The integration of AI, machine learning, and modern web technologies enables real-time, personalized guidance. The system improves accessibility, reduces costs, and enhances decision-making for users. It also provides a strong foundation for future enhancements such as mobile applications, voice-based interaction, and advanced AI integration.

References

[1] A. K. Das and R. Naskar, “A Deep Learning Model for Depression Detection Based on MFCC and CNN Generated Spectrogram Features,” Biomedical Signal Processing and Control, vol. 90, p. 105898, 2024. [2] A. Othmani, et al., “Towards Robust Deep Neural Networks for Affect and Depression Recognition from Speech,” in Pattern Recognition. ICPR International Workshops and Challenges, Springer International Publishing, 2021. [3] A. Saidi, S. B. Othman, and S. B. Saoud, “Hybrid CNN-SVM Classifier for Efficient Depression Detection System,” in Proceedings of the 4th International Conference on Advanced Systems and Emergent Technologies (IC_ASET), IEEE, 2020. [4] H. Negi, et al., “A Novel Approach for Depression Detection Using Audio Sentiment Analysis,” International Journal of Information Systems & Management Science, vol. 1, no. 1, 2018. [5] I. Madhavi, et al., “A Deep Learning Approach for Work-Related Stress Detection from Audio Streams in Cyber-Physical Environments,” in Proceedings of the 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), IEEE, 2020.

Copyright

Copyright © 2026 R. Rajesh, Sankula Sireesha. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.