Smart Complaint Detection and Severity Analysis in Financial Texts

Abstract

Financial institutions receive thousands of complaints daily across digital platforms such as social media, email, and customer service portals. Manual identification, categorization, and resolution of these complaints are slow, inefficient, and error-prone, leading to delayed responses, customer frustration, and missed opportunities for service improvement. The overwhelming volume of unstructured textual data makes it nearly impossible for human agents to process, prioritize, and address every complaint in a timely and effective manner. To address this challenge, this study presents a Multi-Task Financial Complaint Analysis System that leverages a shared RoBERTa architecture to simultaneously perform four classification tasks: aspect classification, severity prediction, emotion detection, and sentiment analysis. The proposed model integrates a pre-trained roberta-base network as the shared encoder with task-specific attention mechanisms and classifier heads for each task. Focal loss and weighted sampling are incorporated to handle class imbalance and enhance model robustness. The framework is trained and evaluated on the Consumer Financial Protection Bureau (CFPB) Consumer Complaint Database containing real-world financial complaints and compared with several baseline approaches. Experimental results demonstrate that the proposed Multi-Task RoBERTa model achieves an overall accuracy of 91.88% and an F1 score of 88.28%, with per-task performance of 76% for aspect classification, 92% for severity prediction, 100% for emotion detection, and 96% for sentiment analysis. For practical deployment, the system is implemented as a Streamlit-based web application capable of real-time complaint analysis with confidence scores and probability distribution visualizations. The proposed framework offers an accurate, interpretable, and reliable solution for automated complaint understanding and prioritization in financial customer service environments.

Introduction

Financial institutions receive a large volume of customer complaints through digital channels such as emails, social media, and customer service portals. These complaints contain important information regarding customer dissatisfaction, service issues, product-related problems, and emotional responses. However, manual processing of such unstructured textual data is time-consuming, inefficient, and prone to errors, resulting in delayed responses and reduced customer satisfaction. Existing automated approaches often focus on individual tasks such as sentiment analysis or basic classification, without addressing the complete understanding of complaints, including product identification, urgency assessment, emotion detection, and sentiment analysis.

This study proposes a Multi-Task Financial Complaint Analysis System that uses advanced artificial intelligence and natural language processing techniques to automatically analyze, categorize, and prioritize financial complaints. The system utilizes transformer-based language models, particularly RoBERTa, combined with multi-task learning to perform four major tasks simultaneously: identifying the financial product or service involved, predicting complaint severity, detecting customer emotion, and classifying overall sentiment. The model was developed using PyTorch, Hugging Face Transformers, and deployed through a Streamlit-based interactive web application.

The proposed framework was trained and evaluated using the CFPB Consumer Complaint Database, which contains real-world financial complaints. Data preprocessing involved dataset organization, stratified division into training (80%), validation (10%), and testing (10%) sets, class imbalance handling through weighted sampling, and text normalization techniques. A shared RoBERTa encoder was used to extract contextual and semantic features, while separate task-specific classifier heads were designed for aspect classification, severity prediction, emotion recognition, and sentiment analysis.

To improve model performance and reliability, several optimization techniques were incorporated, including focal loss, weighted task losses, gradient clipping, early stopping, learning rate scheduling, and learnable task weighting. The system also incorporated explainability features through confidence scores and probability distributions, allowing users to understand prediction outcomes and improve trust in automated decisions.

The Streamlit-based application provides a user-friendly interface where users can enter complaint text, upload complaint files, and receive real-time predictions with visualization of classification results. The multi-task approach enables comprehensive complaint understanding by analyzing multiple aspects simultaneously rather than treating each task independently.

Conclusion

This project successfully developed and implemented a Multi-Task Financial Complaint Analysis System that leverages a shared RoBERTa architecture to simultaneously perform four classification tasks: aspect classification, severity prediction, emotion detection, and sentiment analysis. The system was trained and evaluated on the CFPB Consumer Complaint Database, achieving an overall accuracy of 91.88% and an F1 score of 88.28%, with per-task performance of 76% for aspect classification, 92% for severity prediction, 100% for emotion detection, and 96% for sentiment analysis. The interactive Streamlit dashboard enables real-time complaint analysis with confidence scores and probability distributions, making the system practical for real-world deployment in financial customer service environments. The results validate the effectiveness of the multi-task learning approach, demonstrating that a single shared RoBERTa model can effectively handle multiple related tasks while maintaining high performance across all of them. By addressing the limitations of existing single-task systems and providing a comprehensive, multi-dimensional analysis, this project contributes a valuable, reproducible, and offline-capable solution that can help financial institutions automate complaint processing, prioritize urgent issues, and gain data-driven insights for improving products and services, ultimately making financial customer service more responsive, efficient, and customer-centric through the thoughtful application of artificial intelligence.

References

[1] A. Vaswani et al., \"Attention is All You Need,\" in Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008, 2017. [2] A. Singh, R. Bhatia, and S. Saha, \"Complaint and Severity Identification from Online Financial Content,\" IEEE Transactions on Computational Social Systems, vol. 11, no. 3, pp. 4130–4141, 2024. [3] A. Joshi, A. Kumar, and N. Patel, \"Leveraging Social Media and Multitask Natural Language Processing to Detect Financial Consumer Complaints,\" Economic Computation and Economic Cybernetics Studies and Research, vol. 59, no. 3, 2025. [4] P. Jain et al., \"An intelligent transformer based framework for bilingual financial complaint classification,\" Scientific Reports, vol. 16, 2026. [5] S. Das et al., \"When Words Can\'t Capture It All: Towards Video-Based User Complaint Text Generation,\" in ACM CIKM, 2025. [6] Y. Liu et al., \"RoBERTa: A Robustly Optimized BERT Pretraining Approach,\" arXiv preprint arXiv:1907.11692, 2019. [7] S. Minaee et al., \"Large Language Models: A Survey,\" arXiv preprint arXiv:2402.06196, 2024. [8] C. W. Okonkwo and A. Ade-Ibijola, \"Chatbots applications in education: A systematic review,\" Computers & Education: Artificial Intelligence, vol. 2, p. 100033, 2021. [9] S. Das, A. Singh, S. Saha, and A. Maurya, \"Negative Review or Complaint? Exploring Interpretability in Financial Complaints,\" IEEE Transactions on Computational Social Systems, 2024. [10] S. Das et al., \"Deciphering the Complaint Aspects: Towards an Aspect-Based Complaint Identification Model with Video Complaint Dataset in Finance,\" in IEEE/CVF WACV, 2025, pp. 7195–7204. [11] S. Das et al., \"ExpertMix: Aspect and Severity Detection in Conversational Complaints,\" in Advances in Information Retrieval, 2026. [12] R. R. Selvaraju et al., \"Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,\" in Proceedings of IEEE ICCV, 2017, pp. 618–626. [13] Z. Wang et al., \"From Complaint Narratives to Monetary Relief: A Hybrid Machine Learning Framework for CFPB Consumer Complaints,\" arXiv preprint arXiv:2606.22664, 2026. [14] S. Das et al., \"Knowing What and How: A Multi-modal Aspect-Based Framework for Complaint Detection,\" Advances in Information Retrieval, 2023. [15] M. T. Ribeiro, S. Singh, and C. Guestrin, \"Why Should I Trust You? Explaining the Predictions of Any Classifier,\" in Proceedings of ACM SIGKDD, 2016, pp. 1135–1144.

Copyright

Copyright © 2026 Shravya Koulammagari, B. Kranthi Kiran. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.