Fake News Identification Using Hybrid Transformer Ensemble Approach

Abstract

The rapid spread of misinformation across digital platforms has made fake news detection a critical challenge, as it can influence public opinion, disrupt social stability, and reduce trust in credible information sources. Manual verification is no longer feasible at scale due to the large volume of content generated daily. Existing approaches have explored hybrid architectures combining transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT) with sequential models like Long Short-Term Memory (LSTM) for fake news classification; however, such approaches may have limitations in capturing the diverse linguistic, contextual, and structural patterns present in textual data. To address this limitation, this paper proposes a hybrid transformer-based ensemble model for automated fake news identification using the FakeNewsNet dataset. The proposed system integrates BERT with LSTM for contextual and sequential learning, Robustly Optimized BERT Pretraining Approach (RoBERTa) for improved textual representation, and Light Gradient Boosting Machine (LightGBM) for learning statistical patterns from textual features. Individual models provide strong baseline performance, while the ensemble combines their predictions using a weighted strategy to improve overall accuracy and robustness. Experimental results show that the ensemble model achieves an accuracy of approximately 93%, outperforming the individual constituent models. The system can be applied in real-time news verification platforms to assist users, journalists, and fact-checkers in identifying misleading information more effectively.

Introduction

This work proposes a hybrid transformer-based ensemble model for fake news detection that combines BERT with LSTM, RoBERTa, and LightGBM to improve the accuracy and reliability of classifying news as real or fake. Fake news spreads rapidly through digital platforms and social media, making manual verification impractical. Traditional machine learning models often fail to capture the contextual, sequential, and semantic characteristics of news content, motivating the use of advanced NLP and deep learning techniques.

The proposed system integrates DistilBERT/BERT with LSTM to learn contextual and sequential information, RoBERTa to enhance semantic understanding, and LightGBM to analyze statistical and TF-IDF-based features. SMOTE is applied to balance the training dataset, and a weighted ensemble strategy combines predictions from all three models, resulting in more accurate, robust, and consistent classifications than single-model approaches.

The system uses the FakeNewsNet dataset containing over 23,000 news samples, which undergo preprocessing, tokenization, normalization, and feature extraction before model training. LightGBM processes around 5,015 engineered features, while transformer models use contextual embeddings with a maximum sequence length of 128 tokens.

Compared with existing approaches, the proposed framework offers better handling of class imbalance, improved contextual understanding, stronger generalization, and higher prediction accuracy. A user-friendly web interface enables real-time fake news detection, making the system practical for large-scale and real-world applications.

Conclusion

This paper demonstrates the effectiveness of a hybrid ensemble-based approach for fake news detection by integrating deep learning and machine learning models within a unified framework. The system combines BERT+LSTM, RoBERTa, and LightGBM to capture contextual, sequential, and statistical features from textual data, enabling a comprehensive understanding of news content. The proposed model achieves an overall test accuracy of approximately 93%, along with strong precision, recall, and F1-score values, indicating reliable performance on unseen data. RoBERTa contributes strong contextual understanding, BERT+LSTM enhances sequential pattern learning, and LightGBM captures feature-based statistical patterns, while the weighted ensemble approach improves prediction stability and reduces the limitations of any individual model. The system follows a structured pipeline including preprocessing, SMOTE-based class balancing, feature extraction, and independent model training, ensuring effective learning and generalization, and the implementation of a Streamlit-based interface enables real-time prediction for both single and batch inputs, making the system practical and user-friendly. Overall, this work highlights that combining transformer-based models with traditional gradient-boosting techniques through weighted ensemble aggregation significantly improves fake news detection performance over single-model architectures, while remaining efficient, scalable, and suitable for real-world deployment.

References

[1] A. Aggarwal, A. Chauhan, D. Kumar, M. Mittal, and S. Verma, “Classification of fake news by fine-tuning deep bidirectional transformers-based language model,” EAI Endorsed Trans. Scalable Inf. Syst., vol. 7, no. 27, 2020. [2] A. Agarwal, M. Mittal, A. Pathak, and L. M. Goyal, “Fake news detection using a blend of neural networks: An application of deep learning,” SN Comput. Sci., vol. 1, no. 3, pp. 1–9, 2020. [3] M. Albahar, “A hybrid model for fake news detection: Leveraging news content and user comments,” IET Inf. Secur., vol. 15, no. 2, pp. 169–177, 2021. [4] S. Deepak and B. Chitturi, “Deep neural approach to fake news identification,” Procedia Comput. Sci., vol. 167, pp. 2236–2243, 2020. [5] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv:1810.04805, 2018. [6] C. I. Eke, A. A. Norman, and L. Shuib, “Context-based feature technique for sarcasm identification using deep learning and BERT model,” IEEE Access, vol. 9, pp. 48501–48518, 2021. [7] M. Granik and V. Mesyura, “Fake news detection using naïve Bayes classifier,” in Proc. IEEE First Ukraine Conf. Electr. Comput. Eng. (UKRCON), 2017, pp. 900–903. [8] N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, and D. Lazer, “Fake news on Twitter during the 2016 U.S. presidential election,” Science, vol. 363, no. 6425, pp. 374–378, 2019. [9] H. Jwa, D. Oh, K. Park, J. M. Kang, and H. Lim, “exBAKE: Automatic fake news detection model based on BERT,” Appl. Sci., vol. 9, no. 19, p. 4062, 2019. [10] R. K. Kaliyar, A. Goswami, and P. Narang, “FakeBERT: Fake news detection in social media with a BERT-based deep learning approach,” Multimedia Tools Appl., vol. 80, pp. 11765–11788, 2021. [11] Q. Li and W. Zhou, “Connecting the dots between fact verification and fake news detection,” arXiv:2010.05202, 2020. [12] J. Lin, G. Tremblay-Taylor, G. Mou, D. You, and K. Lee, “Detecting fake news articles using deep learning,” in Proc. IEEE Int. Conf. Big Data, 2019, pp. 1–8. [13] D. Mehta, A. Dwivedi, A. Patra, and M. A. Kumar, “A transformer-based architecture for fake news classification,” Soc. Netw. Anal. Min., vol. 11, no. 1, 2021. [14] T. Mikolov et al., “Advances in pre-training distributed word representations,” in Proc. LREC, 2018. [15] H. Rashkin, E. Choi, J. Y. Jang, S. Volkova, and Y. Choi, “Truth of varying shades: Analyzing language in fake news and political fact-checking,” in Proc. EMNLP, 2017, pp. 2931–2937. [16] N. Ruchansky, S. Seo, and Y. Liu, “CSI: A hybrid deep model for fake news detection,” in Proc. ACM Int. Conf. Inf. Knowl. Manage. (CIKM), 2017, pp. 797–806. [17] K. Shu, D. Mahudeswaran, S. Wang, D. Lee, and H. Liu, “FakeNewsNet: A data repository with news content, social context, and spatiotemporal information,” Big Data, vol. 8, no. 3, pp. 171–188, 2020. [18] M. Sundermeyer, R. Schlüter, and H. Ney, “LSTM neural networks for language modeling,” in Proc. INTERSPEECH, 2012. [19] Z. Yang et al., “Hierarchical attention networks for document classification,” in Proc. NAACL, 2016, pp. 1480–1489. [20] K. Shu, L. Cui, S. Wang, D. Lee, and H. Liu, “dEFEND: A system for explainable fake news detection,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min., 2019, pp. 395–405. [21] N. Rai, D. Kumar, N. Kaushik, C. Raj, and A. Ali, “Fake news classification using transformer-based enhanced LSTM and BERT,” Int. J. Cogn. Comput. Eng., vol. 3, pp. 98–105, 2022. [22] E. Essa, K. Omar, and A. Alqahtani, “Fake news detection based on a hybrid BERT and LightGBM models,” Complex Intell. Syst., vol. 9, pp. 6581–6592, 2023.

Copyright

Copyright © 2026 Mr. E. Chitti Babu, G. Sukanya. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.