AI and NLP for Mental Health Prediction from Social Media: A Decade of Progress, Challenges, and Explainability (2015–2025)

Abstract

Mental health prediction from social media has gained increasing attention due to the growing availability of user data and advancements in artificial intelligence (AI) and natural language processing (NLP). This review examines research from 2015–2025, highlighting datasets, methodologies, and explainability approaches. Early studies applied traditional machine learning with handcrafted features but faced scalability and language limitations. Deep learning and transformer models such as BERT have since achieved superior performance, though challenges of bias, interpretability, and computational cost persist. Dataset analysis reveals a reliance on Reddit (?60%), followed by Twitter (?25%) and smaller contributions from Weibo, Spanish, and Indian corpora, exposing gaps in multilingual coverage. Explainable AI methods (e.g., SHAP, LIME, attention) improve trust and interpretability, yet remain underexplored for non-English contexts. Future work should prioritize inclusive datasets, efficient interpretable models, and multimodal approaches.

Introduction

Mental health has become a significant global public health and socio-economic concern, with approximately one billion people affected worldwide, mainly by depression and anxiety. These conditions are major contributors to disability and economic loss, costing nearly USD 1 trillion annually. Suicide rates remain alarmingly high, with depression as a key risk factor.

Social media platforms now serve as digital mirrors of mental well-being, where users’ language and interaction patterns can reveal psychological states. This has opened new research opportunities using AI and Natural Language Processing (NLP) to analyze social media data for mental health prediction in real time.

The review focuses on AI/NLP methods for detecting mental health issues from social media texts, especially addressing challenges with multilingual data like English, Hindi, and Hinglish, relevant to countries such as India. It also highlights the importance of Explainable AI (XAI) to improve model interpretability for clinical use.

Key developments in approaches (2015–2025):

• Traditional Machine Learning (2015–2018): Used classifiers like SVM, Random Forest, and Naïve Bayes with handcrafted linguistic features, mainly on English data. These models struggled with scalability and multilingual contexts.

• Deep Learning (2018 onward): RNNs, LSTMs, CNNs, and especially Transformer-based models (BERT, RoBERTa) improved accuracy by capturing deeper linguistic and contextual cues. Domain-specific models like MentalBERT further enhanced performance but remain computationally heavy and less interpretable.

• Multilingual and Multicultural Gaps: Most research centers on English datasets. Studies on other languages, including Indian languages and code-mixed text, are limited but growing. Addressing this gap is critical for building inclusive AI systems.

Datasets: Reddit dominates (≈60%) for disorder-specific research, Twitter (≈25%) for short posts and temporal modeling, and platforms like Weibo and mixed-language corpora cover other cultural contexts. Efforts toward multilingual and cross-lingual datasets are emerging.

Conclusion

This review underscores how the field of social media–based mental health prediction has evolved from basic machine learning to advanced deep learning and transformer models over the past decade. Despite significant progress, the field remains limited by linguistic bias, interpretability challenges, and ethical concerns. The dominance of English datasets highlights the urgent need for multilingual, culturally diverse corpora—especially in regions like India, where code-mixed languages are common. While explainable AI techniques have started bridging the trust gap between AI systems and clinical adoption, their integration remains at an early stage, particularly for non-English and resource-constrained settings. Future research should prioritize inclusive datasets, interpretable yet efficient models, and multimodal approaches to ensure scalability, fairness, and clinical relevance. Bridging these gaps will enable AI-powered systems not only to achieve high accuracy but also to make socially impactful contributions in global mental health care.

References

[1] Cuijpers, P., Javed, A., Bhui, K. (2023). The WHO World Mental Health Report: a call for action. British Journal of Psychiatry, 222(6), 227–229. https://doi.org/10.1192/bjp.2023.9 [2] World Health Organization (WHO). (2019). Global strategic direction for mental health. Geneva: WHO. Available at: https://www.who.int/observatories/global-observatory-on-health-research-and-development/analyses-and-syntheses/mental-health/global-strategic-direction [3] Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H., & Eichstaedt, J. C. (2017). Detecting depression and mental illness on social media: an integrative review. Current Opinion in Behavioral Sciences, 18, 43–49. https://doi.org/10.1016/j.cobeha.2017.07.005 [4] De Choudhury, M., Gamon, M., Counts, S., & Horvitz, E. (2013). Predicting depression via social media. Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM), 128–137. [5] Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: a critical review. npj Digital Medicine, 3(43), 1–11. https://doi.org/10.1038/s41746-020-0233-7 [6] Samek, W., Wiegand, T., & Müller, K. R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. ITW 2017 – Information Theory Workshop, 1–6. https://doi.org/10.1109/ITW.2017.8274760 [7] Resnik, P., Armstrong, W., Claudino, L., Nguyen, T., Nguyen, V.A., & Boyd-Graber, J. (2015). Beyond LDA: Exploring supervised topic modeling for depression-related language in Twitter. Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology (CLPsych), ACL, pp. 99–107. [8] Schwartz, H.A., Eichstaedt, J.C., Kern, M.L., et al. (2014). Towards assessing changes in the degree of depression through Facebook. Proceedings of the Workshop on Computational Linguistics and Clinical Psychology (CLPsych), ACL, pp. 118–125. [9] Orabi, A.H., Buddhitha, P., Orabi, M.H., & Inkpen, D. (2018). Deep learning for depression detection of Twitter users. Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology (CLPsych), ACL, pp. 88–97. [10] Ji, S., Zhang, B., Wang, T., Wei, S., & Yu, P.S. (2022). Mental health detection via social media: A survey. ACM Transactions on Intelligent Systems and Technology (TIST), 13(4), Article 69, pp. 1–47. https://doi.org/10.1145/3512730 [11] Ji, S., Li, Y., Huang, H., et al. (2021). MentalBERT: A pretrained language model for mental health text mining. Proceedings of the 20th Workshop on Biomedical Language Processing (BioNLP), ACL, pp. 89–97. https://doi.org/10.18653/v1/2021.bionlp-1.10 [12] Naseem, U., Razzak, I., Musial, K., & Imran, M. (2022). Transformer-based deep intelligent contextual embedding for Twitter sentiment analysis. Future Generation Computer Systems, 113, pp. 58–69. https://doi.org/10.1016/j.future.2020.06.050 [13] Sane, S., & Kumar, A. (2023). A survey on mental health prediction in multilingual social media contexts. Neural Computing and Applications, 35, pp. 14987–15005. https://doi.org/10.1007/s00521-023-08342-4 [14] Coppersmith, G., Dredze, M., Harman, C., Hollingshead, K., Mitchell, M. (2015). CLPsych 2015 Shared Task: Depression and PTSD on Twitter. In: Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology, ACL Anthology. [15] Yates, A., Cohan, A., Goharian, N. (2017). Depression and Self-Harm Risk Assessment in Online Forums. In: Proceedings of EMNLP 2017. Available at: http://ir.cs.georgetown.edu [16] Cohan, A., Desmet, B., Yates, A., Soldaini, L., MacAvaney, S., Goharian, N. (2018). SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions. In: Proceedings of ACL 2018. Also available at: arXiv:1806.05258. [17] Losada, D., Crestani, F., Parapar, J. (2017). eRisk 2017: Overview of CLEF Lab on Early Risk Prediction on the Internet. In: CLEF 2017 Working Notes. CEUR Workshop Proceedings. [18] Turcan, E., McKeown, K. (2019). Dreaddit: A Reddit Dataset for Stress Analysis in Social Media. In: Proceedings of ICWSM 2019. Also available at: arXiv:1905.03013. [19] Yates, A., Cohan, A., Goharian, N. (2018). RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses. In: ACL Workshop on Computational Linguistics and Clinical Psychology. ACL Anthology. [20] Losada, D., Crestani, F., Parapar, J. (2017–2020). The eRisk Series: Early Risk Prediction on the Internet (shared tasks and overviews). In: CLEF Proceedings. CEUR Workshop Proceedings. [21] Louhi et al. (2020–2021). Contextual embedding studies for Reddit-based mental health detection. Published across multiple venues. [22] Multiple authors (2019–2024). BERT-based studies for depression and anxiety detection from social media text. Representative papers across conferences and journals. [23] Weibo Studies (2019–2024). Hierarchical and multimodal transformer approaches for depression detection on Chinese Weibo data. [24] Various authors (2018–2023). User-level aggregation studies: methods aggregating multiple posts for robust user-level predictions. [25] Multimodal works (2020–2024). Combining images and text for improved mental health detection from social media. [26] Early sentiment approaches (2015–2018). Classical machine learning + LIWC-based studies for early depression detection. [27] Cross-evaluation studies (2021–2024). Analyses of domain shift and model generalization challenges in social media-based mental health prediction. [28] Hinglish/India-focused studies (2020–2023). Pilot code-mixed corpora and experiments on Twitter and Hinglish data. [29] Cohan, A. et al. (2023). SMHD-GER and related dataset extensions for standardization of the SMHD benchmark. ACL Anthology. [30] Hybrid deep learning pipelines (2023). Neural hybrid models on the SMHD dataset for multi-disorder classification. [31] Cross-cultural model evaluations (2023–2024). Comparative analyses of models across languages and cultures. Published in NAACL/Findings, ACM Digital Library. [32] SBERT ensemble studies (2024). Embedding ensembles for early detection benchmarks. Semantic Scholar. [33] Hierarchical transformer networks for Weibo (2024). Two-level user modeling for Chinese depression detection tasks. [34] Spanish depression detection studies (2019–2023). Experiments on Spanish Twitter corpora with transfer learning. [35] India-centric BERT approaches (2024). BERT with optimizer tuning for Hinglish mental health datasets. [36] Transformer pipelines in eRisk shared tasks (2020). BERT-based submissions tuned for early detection. [37] Baseline anorexia and self-harm detection datasets (CLEF 2019). CLEF shared task baselines. [38] Explainability & interpretability studies (2018–2023). SHAP, LIME, and attention-based methods for mental health prediction. [39] Temporal modeling studies (2018–2022). RSDD-Time and related temporality-focused analyses. [40] Multilingual transformer studies (2021–2024). mBERT, XLM-R, MuRIL applied for multilingual mental health prediction. [41] Clinician-annotated datasets (2024–2025). ReDSM5 and DSM-5 symptom-level datasets for clinically aligned prediction. [42] Behavioral and emoji-based studies (2019–2024). Fusion of temporal/behavioral cues with text features for better prediction. [43] Review and meta-analyses (2020–2024). Surveys summarizing AI/NLP methods, ethical considerations, and limitations in mental health prediction. [44] Cohan, A., Desmet, B., Yates, A., Soldaini, L., MacAvaney, S., Goharian, N. (2018). SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions. In: Proceedings of ACL 2018. arXiv:1806.05258. [45] Losada, D., Crestani, F., Parapar, J. (2017). eRisk 2017: Overview of CLEF Lab on Early Risk Prediction on the Internet. In: CLEF 2017 Working Notes, CEUR-WS. [46] Turcan, E., McKeown, K. (2019). Dreaddit: A Reddit Dataset for Stress Analysis in Social Media. In: Proceedings of ICWSM 2019. arXiv:1905.03013. [47] Ates, A., Cohan, A., Goharian, N. (2018). RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses. In: ACL Workshop on Computational Linguistics and Clinical Psychology, ACL Anthology. [48] Benton, A., Mitchell, M., Hovy, D.: Multimodal mental health analysis. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). (2017). https://doi.org/10.18653/v1/D17-1185 [49] Ji, S., Zhang, B., Wang, T., Wei, S., Yu, P.S. (2022). Mental health detection via social media: A survey. ACM TIST, 13(4), Article 69. https://doi.org/10.1145/3512730 [50] Ji, S., et al. (2021). Suicidal ideation detection via contextual embedding and hierarchical attention. Information Processing & Management, 58(3), 102542. https://doi.org/10.1016/j.ipm.2020.102542 [51] ibeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM (2016). https://doi.org/10.1145/2939672.2939778 [52] Kaur, H., Mangat, V.: Depression detection on social media using machine learning and LIME. Journal of Ambient Intelligence and Humanized Computing (2020). https://doi.org/10.1007/s12652-020-01845-w [53] Lundberg, S.M., Lee, S.-I.: A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems (NeurIPS) (2017). https://doi.org/10.48550/arXiv.1705.07874 [54] Yang, Z., Dai, Z., Yang, Y., et al.: Attention is all you need in NLP: Hierarchical attention networks for document classification. NAACL (2016). https://doi.org/10.18653/v1/N16-1174 [55] Jain, S., Wallace, B.C.: Attention is not explanation. NAACL-HLT (2019). https://doi.org/10.48550/arXiv.1902.10186 [56] Aldarwish, M., & Ahmad, H.F. (2017). Predicting depression levels using social media posts. IEEE ICT. https://doi.org/10.1109/ICT.2017.7976186 [57] Pestian, J.P., et al. (2017). Machine learning classification of suicidal and non-suicidal patients. Biomedical Informatics Insights, 10. https://doi.org/10.1177/1178222617725071 [58] Matero, M., et al. (2019). Suicide risk assessment with multi-level attention models. CLPsych Workshop. https://doi.org/10.18653/v1/W19-3009 [59] Sharma, E., & De Choudhury, M. (2021). Measuring and mitigating language biases in mental health classification. CHI. https://doi.org/10.1145/3411764.3445423 [60] Bentum, J., et al. (2022). Combining attention and SHAP for interpretable mental health detection. Expert Systems with Applications, 198, 116792. https://doi.org/10.1016/j.eswa.2022.116792 [61] Liang, H., et al. (2023). Hybrid explainability methods for transformer-based mental health detection. Information Sciences, 626, 441–456. https://doi.org/10.1016/j.ins.2023.03.016 [62] Anonymous ReDSM5 Study (2025). Clinically annotated DSM-5 symptom detection with explainable models. ArXiv Preprint (under review)

Copyright

Copyright © 2025 Ranjeet Singh Thakur, JP Singh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.