Fake News Detection using BERT & ROBERTA

Abstract

The fake news detection system leverages advanced transformer-based architectures—BERT and RoBERTa—to accurately identify and classify misinformation in textual content. Unlike traditional NLP approaches, these pretrained language models excel at capturing contextual nuances, semantics, and deeper linguistic patterns across long-range dependencies in text. Fine-tuned on large-scale datasets containing both realandfakenewsarticles, thesystemiscapableofdiscerningsubtlepatternsandinconsistenciesoftenpresent in manipulated or misleading narratives. BERT’s bidirectional encoding and RoBERTa’s optimized training strategies contribute to superior performance in understanding the complexity of natural language, ensuring precise and reliablefake news detection. Thebackend of the system isbuiltusing Flask, providing efficientAPI endpointsthat allowusersto input text data. Uponsubmission, themodel evaluatestheinputand classifiesit as either fake or real, accompanied by a confidence score to reflect the likelihood of misinformation.To maintain robustness and adaptability, the system supports continuous learning, allowing the models to be retrained with newdatatokeeppacewithevolvingdeceptivetechniquesinnewsdissemination.Modelperformanceisevaluated using key metrics such as accuracy, precision, recall, and F1-score, ensuring that the system remains both dependable and scalable for real-world applications. This makes the proposed framework highly effective in combating the spread of fake news across digital platforms.

Introduction

The paper addresses the growing challenge of detecting deepfake content and fake news due to advanced generative models producing highly realistic, deceptive media. It proposes DeepDetect, a deepfake detection system that leverages Vision Transformers (ViTs) fine-tuned on large datasets, combined with a Flask backend for real-time image processing and user interaction. The system is evaluated using key performance metrics, demonstrating robustness against manipulated content.

The project also focuses on fake news detection using transformer-based NLP models, specifically BERT and RoBERTa, to overcome challenges like lack of large labeled datasets, evolving misinformation techniques, and diverse writing styles across domains. These models provide deep contextual understanding, outperforming traditional methods in classifying news as real or fake.

A Flask-based web interface enables real-time input and classification of news articles, providing confidence scores and supporting continuous learning to adapt to new misinformation patterns. The system architecture involves dataset preprocessing, fine-tuning transformer models, and real-time classification. Performance is assessed using accuracy, precision, recall, F1-score, confusion matrices, and ROC curves.

Conclusion

Intoday\'sdigitallandscape,thewidespreaddisseminationoffakenewshasemergedasaseriousthreattopublic trust, socialharmony, anddemocraticprocesses. Withtherapidgrowthofonline mediaplatforms,the need for accurateandautomatedfakenewsdetectionsystemshasneverbeenmorecritical.Tocombatthischallenge,we developedarobustsolution:afakenewsdetectionmodelleveragingBERTandRoBERTa,twostate-of-the- art transformer-based NLP models. These architectures are capable of capturing contextual nuances and semantic meaning in text, making them well-suited for distinguishing between real and fabricated news content.Our system was trained and evaluated on benchmark datasets and demonstrated high accuracy, precision, recall, and F1-score, confirming its effectiveness in detecting misinformation. The model is integratedintoaFlask-basedwebinterface,enablinguserstoinputnewstextandinstantlyreceiveaprediction, accompanied bya confidence score to ensure transparency and trust.

References

[1] J.Devlin,M.-W.Chang,K.Lee,andK.Toutanova,“BERT:Pre-trainingofDeepBidirectional Transformers for Language Understanding,” Proc. NAACL-HLT, pp. 4171–4186, 2019. [2] Y.Liuetal.,“RoBERTa:ARobustlyOptimizedBERTPretrainingApproach,”arXivpreprint arXiv:1907.11692, 2019. [3] N.Ruchansky,S.Seo,andY.Liu, “CSI:AHybridDeepModelforFakeNewsDetection,”inProc.CIKM,pp.797–806,2017. [4] S.Thorneetal.,“FEVER:ALarge-scaleDatasetforFactExtractionandVerification,”Proc.NAACL-HLT,pp.809–819,2018. [5] H.Zhang,P.Zhang,andY.Yuan,“FAKEDETECTOR:EffectiveFakeNewsDetectionwithDeep Diffusive Network Model,” IEEE T. Knowl. Data Eng., vol. 33, no. 5, pp. 2225– 2238, May2021. [6] A. Hanselowskiet al., “Retrospective Fake News Detection Using BERT,” Proc. NLP4IF@EMNLP, pp. 1–8, 2019. [7] S.KarimiandJ.Tang,“DeepLearningforDetectingFakeNewsinSocialMedia,”IEEEAccess,vol.8,pp. 90594–90601, 2020. [8] M.Zhou,W.Shu,D.Zhang,andJ.Wu,“FakeNewsDetectionviaNLPEnhancedwithTransformer-based Architectures,” in Proc. IJCAI, pp. 4532–4538, 2021. [9] D.Waddenetal.,“FactorFiction:VerifyingScientificClaims,”EMNLP,pp.7534–7550, 2020. [10] R.Shu,A.Sliva,H.Wang,andB.Liu,“FakeNewsDetectiononSocialMedia:ADataMining Perspective,” SIGKDD Explorations, vol. 19, no. 1, pp. 22–36, 2017. [11] S.Singhania,N.Fernandez,andS.Rao,“3HAN:ADeepNeuralNetworkforFakeNewsDetection,”IEEEIntelligentSystems, vol. 35,no. 4, pp. 45–50,2020. [12] S.Vaswaniet al.,“AttentionIsAllYouNeed,”NeurIPS,pp.5998–6008,2017. [13] A.Kaliyar,A.Goswami,andP.Narang,“DeepFakE:ImprovingFakeNewsDetectionUsingEntity Recognition and Emotion Classification,” IEEE Access, vol. 8, pp. 100947– 100958, 2020. [14] B.PangandL.Lee,“OpinionMiningandSentimentAnalysis,”Found.TrendsInf.Retr.,vol. 2,no.1–2,pp.1–135,2008. [15] Z.Wang,C.Li,W.Zhang,andC.Cao,“CombiningBERTwithKnowledgeGraphforFakeNews Detection,” IEEE Access, vol. 9, pp. 148514–148524, 2021. [16] S. Jwa, H. Oh, K. Park, and M. Cha, “ExBAKE: Explainable FakeNewsDetectionUsing Knowledge- Enhanced BERT,” in Proc. ACL, pp. 317–322, 2019. [17] W.Y.Wang,“Liar,LiarPantsonFire:ANewBenchmarkDatasetforFakeNewsDetection,”Proc.ACL, vol. 2, pp. 422– 426, 2017. [18] Y.ZhouandR.Zafarani,“ASurveyofFakeNews:FundamentalTheories,DetectionMethods,and Opportunities,” ACM Comput. Surv., vol. 53, no. 5, pp. 1–40, 2020. [19] K. Shu, D. Mahudeswaran, and H. Liu, “FakeNewsNet: A Data Repository with News Content, SocialContext, and Dynamic Information for Fake NewsResearch,” Big Data, vol. 8, no. 3, pp. 171–188, 2020. [20] N. A. Aslam, T. Nazir, and F. Saeed, “Fake News Detection using RoBERTa and Ensemble Learning,” in Proc. ICAC, pp. 237–242, 2021.

Copyright

Copyright © 2025 Mrs. T Poovozhi, Syed Afsar Ahamed, Thota Sankeerth, Suram Suresh Reddy, Tempalli Ganesh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.