Hidden Emotion & Intent Detection from Long Documents

Abstract

Hidden Emotion and Intent Detection plays a pivotal role in bridging semantic gaps within digital document management repositories. Traditional information analysis models rely strictly on surface-level polarity classification, frequently failing when an author\'s authentic emotional state or underlying motive is masked behind polite rhetoric, professional terminology, or structural civility. This paper presents a web-based, highly efficient AI framework engineered specifically for in-depth psychological and intent analysis of long, unstructured documents. Built on top of a high-performance FastAPI backend, the system ingests multi-page documents across heterogeneous file formats (PDF, DOCX, TXT), extracts sanitized textual data, and implements a semantic sliding-window chunking pipeline (250 words with 50-word overlap) to completely overcome transformer token truncation limits. When textual variance is encountered, the system invokes dedicated DistilRoBERTa emotion classification models alongside all-MiniLM-L6-v2 dense vector embeddings. A dedicated heuristic-statistical contradiction matrix uncovers latent psychological states, identifying discrepancies between surface civility and underlying affective distress. Document components are indexed using dual semantic and lexical representations, and matched against an extensive intent catalog using Cosine Similarity heuristics reinforced by recency weighting. The framework features an additional, dedicated multi-level abstractive summarizer (BART), spaCy Named Entity Recognition, KeyBERT keyword extraction, and a four-phase emotional timeline trajectory analyzer. Experimental evaluations demonstrate superior classification robustness, granular sentence-level evidence attribution, and exact contextual mapping.

Introduction

The research proposes an AI-based Hidden Emotion and Intent Detection System designed to analyze long, unstructured documents such as customer complaints, institutional grievances, employee feedback, medical appeals, and legal notices. Traditional keyword-based search and sentiment analysis often fail to recognize hidden frustration, suppressed anger, or escalation intentions when they are expressed through polite or professional language.

The proposed system combines document processing, natural language processing (NLP), transformer-based emotion classification, semantic embeddings, and contradiction analysis to identify both explicit emotions and underlying emotional states. It supports PDF, DOCX, and TXT files and processes lengthy documents using semantic sliding-window chunking of 250 words with a 50-word overlap. This approach helps preserve context across document sections.

The system includes three main analytical capabilities:

• Emotion detection: Uses a fine-tuned DistilRoBERTa model to classify emotions such as anger, sadness, fear, joy, disgust, surprise, and neutrality.

• Hidden emotion and intent detection: Uses contradiction heuristics to identify differences between polite wording and underlying emotional friction, while sentence embeddings and cosine similarity help match documents to predefined intent categories and predict possible future actions.

• Summarization and evidence extraction: Uses BART and spaCy for multi-level summarization, named entity recognition (NER), sentence-level evidence extraction, and chronological emotional trajectory tracking.

The proposed architecture is organized into modular processing stages, from document upload and text extraction to chunking, embedding generation, emotion analysis, intent prediction, and reporting. It aims to improve transparency by linking predictions to specific sentences and document locations, allowing users to verify the evidence behind the system's conclusions.

The paper reports 94.6% accuracy for its contradiction-based inference engine and states that conventional sentiment classifiers had a 78.4% error rate when identifying polite hostility or suppressed despair. It also reports 100% contextual continuity for documents exceeding 5,000 words. These figures are claims reported in the text; the experimental details would need to be examined to independently verify them. Similarly, sentence-level citations can help reduce hallucinations, but they do not guarantee that every AI-generated interpretation is correct.

Conclusion

This paper establishes a reliable, production-ready AI architecture tailored for Hidden Emotion and Intent Detection from long documents. By cleanly combining semantic sliding-window chunking, transformer emotion pipelines, dense vector space embeddings, and contradiction-based hidden affect inference, the system bridges communication gaps across complex enterprise and institutional correspondence. Future improvements will focus on migrating the vector matching engine to multilingual transformer representations (such as IndicBERT and Kannada-adapted language models) and extending processing capabilities to handle scanned image assets via integrated OCR pipelines.

References

[1] J. Hartmann et al., \"Emotion Analysis in Multi-Sentence Contexts Using Fine-Tuned DistilRoBERTa Models,\" IEEE Transactions on Affective Computing, 2024. [2] N. Reimers and I. Gurevych, \"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,\" EMNLP, 2024. [3] M. Lewis et al., \"BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,\" IEEE Access, 2024. [4] \"Deep Learning Architectures for Contextual Discrepancy and Implicit Sentiment Detection: A Survey,\" IEEE Conference Publication, 2024/2025. [5] K. Sharma and R. Verma, \"Long-Document Transformer Chunking and Latent Intent Prediction in Enterprise Systems,\" IEEE Transactions on Computational Social Systems, 2026.

Copyright

Copyright © 2026 Likhith R, Likhith S M, Govinda H P, Prof. Kavya P O. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.