Automatic Text Summarization using Long Short-Term Memory (LSTM)

Abstract

Text summarization is the process of automatically generating a shorter version of a given text while retaining its important information. Long Short-Term Memory (LSTM) is a type of recurrent neural network that is commonly used in natural language processing tasks such as text summarization. LSTM networks have a memory component that allows them to remember important information from the input text, which enables them to generate a more concise and relevant summary of the original text. LSTM networks can be trained on a large corpus of text data, and they can be fine-tuned for specific applications such as summarization. Overall, LSTM networks are a powerful tool for text summarization, as they can effectively capture the long-term dependencies in natural language data and produce high-quality summaries. Long Short-Term Memory (LSTM) is a type of recurrent neural network (RNN) that is able to effectively capture long-term dependencies in sequential data. LSTMs are composed of memory cells, input gates, forget gates, and output gates, which allow the network to selectively remember and forget information over time. This makes LSTMs well-suited for tasks such as language modeling and time series prediction. Despite their ability to handle complex sequential data, LSTMs are still subject to the vanishing gradient problem, which can limit their performance on longer sequences. However, recent advancements in LSTM architecture have helped to alleviate this issue.

Introduction

Automatic text summarization is an essential NLP task that condenses large texts into brief, coherent summaries. It mainly uses two methods: extractive (selecting key sentences verbatim) and abstractive (generating paraphrased summaries). Deep learning, especially LSTM-based sequence-to-sequence models with attention mechanisms, has significantly advanced summarization by better capturing context and sequential dependencies. However, LSTMs face challenges like high computational cost, dependence on large datasets, and limitations in deep semantic understanding compared to transformer models.

The paper reviews related work including fuzzy rule-based methods, deep learning approaches for legal texts, multimodal summarization combining images and text, RBM and fuzzy logic hybrid methods, and transformer-based abstractive summarization which achieves state-of-the-art results but with high resource needs.

The proposed work implements a bi-directional LSTM with multi-head attention in a Seq2Seq framework, using datasets like Amazon Fine Food Reviews, CNN/DailyMail, and Gigaword for training and evaluation. This approach improves summarization quality, achieving high ROUGE scores and outperforming traditional LSTM and other baseline models while maintaining lower computational costs than transformer-based systems.

Experimental results show that the combination of Spacy’s NLP tools with LSTM yields the best precision, recall, and F1-scores compared to other summarization tools like Sumy, Gensim, and NLTK, highlighting the effectiveness of deep learning integrated with advanced NLP preprocessing.

Conclusion

In this study, we introduced an advanced text summarization model that synergizes Bi-directional Long Short-Term Memory (Bi-LSTM) networks with multi-head attention mechanisms within a sequence-to-sequence (Seq2Seq) framework. This architecture adeptly captures contextual dependencies in both forward and backward directions, while the attention mechanism enhances the model\'s focus on salient information, leading to the generation of coherent and contextually relevant summaries. Utilizing datasets such as Amazon Fine Food Reviews, CNN/DailyMail, and Gigaword, our model demonstrated superior performance, achieving a precision of 0.96, recall of 0.94, F1-score of 0.95, and a ROUGE score of 0.93, outperforming traditional LSTM models lacking attention mechanisms. Despite these promising results, challenges persist, including the model\'s reliance on large, high-quality datasets and substantial computational resources. Future research directions include integrating transformer-based architectures to further enhance semantic understanding, employing reinforcement learning for summary optimization, and developing more robust evaluation metrics that assess coherence and relevance beyond lexical overlap. Such advancements aim to refine the summarization process, making it more efficient and adaptable across various domains, thereby addressing the growing demand for effective information distillation in the digital age.

References

[1] Saraswathi, R. V., Chunchu, R. V., Kunchala, S., Varun, M., Begari, T., & Bodduru, S.(2022). A Deep Learning Model for Text Summarization. [2] Li, M., Xing, T., Fu, R., & Yin, F. (2021). Research on Text Summarization Generation Based on LSTM and Attention Mechanism. College of Information and Communication Engineering, Communication University of China, Beijing, China. [3] Zhang, Y., Liu, W., & Chen, X. (2022). Attention-based LSTM for Automatic Text Summarization. Journal of Computational Linguistics, 35(3), 345-360. [4] Kumar, P., Rao, T., & Singh, A. (2023). Exploring Reinforcement Learning in LSTM for Text Summarization. [5] Li, M., Wang, X., & Chen, Z. (2023). Improving Abstractive Summarization with Dual LSTM Networks. [6] Ghanem, F. A., Padma, M. C., Abdulwahab, H. M., & Alkhatib, R. (2025). Deep Learning-Based Short Text Summarization: An Integrated BERT and Transformer Encoder–Decoder Approach. [7] Huang, J., Wu, W., Li, J., & Wang, S. (2023). Text Summarization Method Based on Gated Attention Graph Neural Network. [8] Zheng, C., Zhang, K., Wang, H. J., Fan, L., & Wang, Z. (2021). Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization. [9] See, A., Liu, P. J., & Manning, C. D. (2017). Get To The Point: Summarization with Pointer-Generator Networks. arXiv preprint arXiv:1704.04368. [10] Chouikhi, H., & Alsuhaibani, M. (2022). Deep Transformer Language Models for Arabic Text Summarization: A Comparison Study. | [11] Gupta, A., Chugh, D., Anjum, & Katarya, R. (2021). Automated News Summarization Using Transformer. [12] Ramesh, D., Kothandaraman, D., Chegoni, R., Mohmmad, S., & Pasha, S. N. (2023). Abstractive Text Summarizer Using Neural Networks Algorithms. [13] Xu, Z., & Zhu, J. (2022). Deep Hierarchical LSTM Networks with Attention for Video Summarization. Computers & Electrical Engineering, 97, 107595. [14] Li, J., Zhang, C., & Chen, X. (2021). Text Summarization Based on Multi-Head Self-Attention Mechanism and Pointer Network. [15] El-Kassas, W. S., Salama, C. R., Rafea, A. A., & Mohamed, H. K. (2021). Automatic Text Summarization: A Comprehensive Survey.

Copyright

Copyright © 2025 P. Sushanth, Dr. K Santhi Sree . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.