A Comprehensive Study on Hybrid-Architecture Fusion for Robust and Explainable Deepfake Detection

Abstract

Deepfake technology has reached a point where manipulated facial videos exhibit highly realistic visual quality, making traditional detection methods increasingly ineffective. The growing sophistication of generative models is outpacing the capabilities of contemporary deepfake detection solutions; as a consequence, there is an even greater necessity to create methods that are more adaptive. Generally speaking, present detection solutions use compression artifacts from the original media and abnormalities caused by noise to identify a deepfake; however, these two predominant methodologies severely restrict the use of current solutions in natural circumstances. Existing models often suffer from generalization issues across datasets, are limited in capturing both local texture abnormalities and global contextual inconsistencies, and are usually not interpretable, further limiting their application in security-critical environments. This study provides an in-depth investigation of a hybrid deepfake detection framework that integrates efficient convolutional architectures and transformer-based models with XAI. The proposed dual-pathway design uses either EfficientNetV2 or MobileViT for the extraction of fine-grained local features and Swin Transformer (Tiny) to model long-range dependencies and global spatial relationships. The design intends to combine this set of complementary architectures for improving robustness against diverse manipulation techniques and ensuring that the outcomes remain computationally efficient for possible real-time deployments. Explainability modules such as Grad-CAM and LIME are integrated into the system to ensure that the model decisions are transparently interpreted visually, thus overcoming the limitations of black-box deep learning systems. This fundamental study sets a theoretical and methodological basis for establishing a reliable, generalizable, interpretable deepfake detection framework that would be expanded and fully implemented in later phases of the project.

Introduction

The rise of AI and deep learning has enabled the creation of highly realistic deepfakes, which can manipulate images and videos to convincingly alter appearances. While this technology has legitimate applications in film, entertainment, and virtual reality, it also poses significant ethical and security risks, including misinformation, identity theft, and political manipulation. Detecting deepfakes is increasingly challenging because generative models like GANs and diffusion models produce near-perfect replicas of human faces, which can deceive both humans and conventional detection systems.

Traditional single-model detection methods often fail to capture both local texture anomalies and global contextual inconsistencies, and most operate as “black boxes,” offering no transparency in decision-making. To address these issues, hybrid detection frameworks have been proposed that combine convolutional neural networks (EfficientNetV2-S, MobileViT-S) with transformer-based models (Swin Transformer-Tiny) to leverage complementary texture- and context-based feature extraction. These hybrid systems use score-level fusion to improve accuracy, robustness, and generalization across multiple deepfake datasets.

Explainable AI (XAI) techniques like Grad-CAM and SHAP are incorporated to visualize decision regions and improve model interpretability and trustworthiness. Experimental results show that hybrid models outperform single architectures, achieving accuracy up to 95.2%, F1-scores of 0.94, and ROC-AUC of 0.96. Individually, EfficientNetV2-S, MobileViT-S, and Swin Transformer-Tiny also achieve strong results, with Swin Transformer achieving up to 97.9% accuracy on benchmark datasets. These hybrid, interpretable frameworks provide a high-performance, trustworthy, and scalable solution for deepfake detection, balancing local detail recognition with global contextual understanding for real-world applications in digital media verification and cybersecurity.

Conclusion

The development of the Hybrid Deepfake Detection Framework is a major progress toward mitigating the growing risks of manipulated media. By integrating efficient convolutional architectures (EfficientNetV2 or MobileViT) together with transformer-based models - Swin Transformer Tiny - and incorporating Explainable AI. This project brings together techniques that establish a comprehensive solution balancing accuracy, computational efficiency, and interpretability are key properties in real-world deployment in security-sensitive applications. The Dual-Pathway Architecture has succeeded in utilizing the complementary strengths of CNNs and transformers can detect local texture-level artifacts as well as simultaneous global contextual inconsistencies. This holistic approach strengthens the model\'s capability to Generalize across diverse deepfake generation techniques and datasets, overcoming limitations. Single-architecture approaches, which often struggle with novel manipulation methods or cross dataset performance degradation. The integration of Explainable AI techniques, especially Grad-CAM and LIME, addresses one of the fundamental gaps in deep learning-based detection systems: lack of transparency of decision-making processes. By visually explaining and emphasizing, regions of suspicion, the system enhances user trust, facilitates error analysis and allows Continuous model refinement through human oversight. This interpretability is particularly of value in the legal, forensic, and journalistic fields for evidence justification and Accountability is paramount. Implementation completely in Google Colab shows its usability and scalability of modern cloud-based development environments, eliminating barriers related to local hardware requirements, and enabling rapid prototyping and experimentation. The modular architecture, full preprocessing pipelines, and strong training strategies ensure that the framework can easily be extended, adapted, or integrated into larger systems which address multi-media authentication challenges.

References

[1] Deng, X., Li, H., Zhu, J., & Sun, Z. (2022). Deepfake video detection based on EfficientNet-V2 network. Journal of Visual Communication and Image Representation, 86, 103556. https://doi.org/10.1016/j.jvcir.2022.103556 [2] Wang, Y., & Lu, H. (2023). Face forgery detection using an improved MobileViT network with coordinate attention and GELU activation. IEEE Access, 11, 55321–55332. https://doi.org/10.1109/ACCESS.2023.3268471 [3] Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., … Guo, B. (2021). Swin Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 10012–10022. [4] Afchar, D., Nozick, V., Yamagishi, J., & Echizen, I. (2018). MesoNet: A compact facial video forgery detection network. IEEE International Workshop on Information Forensics and Security (WIFS), 1–7. [5] Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019). FaceForensics++: Learning to detect manipulated facial images. IEEE/CVF International Conference on Computer Vision (ICCV), 1–11. [6] Dolhansky, B., Howes, R., Pflaum, B., Baram, N., & Ferrer, C. C. (2020). The DeepFake Detection Challenge dataset. arXiv preprint arXiv:2006.07397. [7] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 618–626. [8] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you?: Explaining the predictions of any classifier. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. [9] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 4765–4774. [10] Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4401–4410. [11] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … Bengio, Y. (2014). Generative adversarial nets. Advances in Neural Information Processing Systems, 2672–2680. [12] Chattopadhyay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018). Grad-CAM++: Improved visual explanations for deep convolutional networks. IEEE Transactions on Image Processing, 30, 2947–2958. [13] Makridis, G., Boullosa, P., & Sester, M. (2023). Enhancing explainability in mobility data science through a combination of methods. GeoXAI Workshop Proceedings, 3(1), 1–1. [14] Poggio, T., Serre, T., & Mutch, J. (2011). Visual object recognition. Synthesis Lectures on Artificial Intelligence and Machine Learning, 5(2), 1–181. [15] Shotton, J., Blake, A., & Cipolla, R. (2008). Object detection by global contour shape. Pattern Recognition, 41(12), 3736–3748. [16] Sudderth, E. B., Torralba, A., Freeman, W. T., & Willsky, A. S. (2009). Unsupervised learning of probabilistic object models for classification, segmentation, and recognition using knowledge propagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(10), 1747–1774.

Copyright

Copyright © 2025 Dr. S Gunasekaran , Amritha Devadasan, Archana J, Gauthami Ganesan, Kripa S, Nandhana S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.