Leveraging ResNet and U-Net Architectures for Accurate Image Forgery Detection and Localization: A Comprehensive Survey

Abstract

The area of image forgery localization and detection (IFDL) has received considerable attention because of the increased manipulations in the digital age facilitated by powerful generative AI technology. This survey focuses on recent trends in deep learning techniques, especially multimodal, explainable, and generalizable models built up to 2025. U-Net variants and transformer-based models are the main developments, focusing on forgery trace detection. Deep learning models, especially ResNet and U-Net, have improved image forgery detection and localization due to better feature extraction and semantic segmentation. The review incorporates state-of-the-art approaches employing these models and carries out a critical appraisal of design, evaluation metrics, limitations, and possible extensions. The FakeShield framework brings in a trend change by embracing multi-modal LLMs for explainable detection and localization. New approaches like MMTD-Set have provided a greater cross-domain adaptability during model training. Complementary studies investigate novel paradigms to enhance localization accuracy and robustness against anti-forensic attacks. In-depth analyses emphasize the use of complementary forensic analyses when considering real-life scenarios. This survey is a complete guide for research and practitioners working on designing next-generation image forgery detection systems that are accurate yet explainable.

Introduction

• Image forgery is increasingly common due to advanced manipulation tools and generative models.

• Impacts critical sectors like journalism, law, and social media.

• Reliable, interpretable, and automated forgery detection systems are vital to maintain the integrity of visual content.

2. Key Deep Learning Architectures

???? Residual Networks (ResNet)

• Uses shortcut (residual) connections to combat vanishing gradients.

• ResNet-50 is popular for deep feature extraction.

• Applied in:

  • Copy-move forgery detection

  • Federated learning for face forgery with privacy

  • Hybrid models like TASPP-UNet for enhanced localization

• Achieves >95% accuracy on datasets like CASIA and MICC-F220.

???? U-Net and Variants

• An encoder-decoder architecture with skip connections ideal for pixel-level segmentation.

• Applied to:

  • Splicing and copy-move forgeries

  • Grasshopper-optimized U-Net for social media forgery

  • Dense U-Net, RRU-Net (with ringed residuals), and attention-based models (e.g., Circular U-Net, UCM-Net)

• Supports fine-grained localization and high adaptability.

???? Hybrid U-Net + ResNet Architectures

• Combine ResNet encoders for feature extraction with U-Net for segmentation.

• Examples:

  • Residual U-Net with VGG16 encoder

  • Hierarchical progressive U-Net with ResNet backbones

  • Context-aware models that improve detection by leveraging image semantics

???? Ringed Residual U-Net (RRU-Net)

• Designed for image splicing forgery.

• Inspired by human cognition:

  • Residual Propagation for key feature retention

  • Residual Feedback with attention to highlight tampered regions

  • Ringed Structure for cyclic learning and strong performance

• Outperforms other models in precision, recall, and F1 score.

???? Residual Transformer (ResTran)

• Merges ResNet (local features) with Transformer decoder (global context).

• Excels in multi-tampering detection.

• Strong performance on datasets like CASIA, NIST, IMD2020, especially for splicing forgeries.

3. Forgery Detection Paradigms

A. Classification vs. Localization

• ResNet excels at binary classification (real vs. fake).

• U-Net specializes in pixel-level localization.

• Hybrid models merge both for more effective and interpretable results.

B. Transfer Learning

• Uses pre-trained models (e.g., ResNet-50 from ImageNet) to overcome limited data.

• Boosts accuracy and reduces training time.

C. Federated Learning

• Enables privacy-preserving decentralized model training.

• Maintains data security without sacrificing accuracy.

D. Integration of Classical Methods

• Combines traditional techniques (e.g., Error Level Analysis, edge detection) with deep learning for enhanced performance.

4. Benchmark Datasets & Metrics

A. Popular Datasets

B. Key Evaluation Metrics

• Accuracy, Precision, Recall, F1-Score – for classification

• IoU, Dice Coefficient – for segmentation

• Robustness Tests – simulate JPEG compression, brightness changes, etc.

???? Example Performance on CASIA2:

5. Emerging Trends & Future Directions

A. Explainable AI (XAI)

• Adds transparency to detection systems, improving forensic and legal admissibility.

B. Transformers & Attention Mechanisms

• Vision Transformers (ViT) and multi-scale attention enhance context modeling and detection granularity.

C. Multi-Modal & Hybrid Approaches

• Combine:

  • Handcrafted forensic features

  • Metadata and sensor data

  • Deep features for robust and comprehensive detection

D. Federated & Privacy-Preserving Learning

• Supports data protection regulations while allowing large-scale collaborative model training.

E. Benchmark Standardization

• Calls for unified benchmarks to fairly evaluate models and encourage reproducibility across studies.

Conclusion

This study highlights that ResNet and U-Net structures, separately and in hybrid versions, form the foundation of today\'s image forgery detection and localization technology. Their mutual strength—deep feature extraction and sharp segmentation—remedy central problems in detecting progressively advanced digital forgeries. Emerging trends like transformer incorporation, explainable AI, and privacy-protection federated learning are about to further advance the discipline. Future work will need to emphasize generalizability to unobserved forgery methods, adversarial robustness, and interpretable model development appropriate for forensic and legal contexts. Enlargement and normalization of a variety of different benchmark datasets will be instrumental in furthering model evaluation and deployment preparedness.

References

[1] N. Kashyap, P. Yadav, Nikita and M. A. Kaushal, \"Deep Learning Strategies for Effective Image Forgery Detection and Localization,\" 2024 International Conference on Computer, Electronics, Electrical Engineering & their Applications (IC2E3), Srinagar Garhwal, Uttarakhand, India, 2024, pp. 1-5, doi: 10.1109/IC2E362166.2024.10827254. [2] A. Meepaganithage, S. Rath, M. Nicolescu, M. Nicolescu and S. Sengupta, \"Image Forgery Detection Using Convolutional Neural Networks,\" 2024 12th International Symposium on Digital Forensics and Security (ISDFS), San Antonio, TX, USA, 2024, pp. 1-6, doi: 10.1109/ISDFS60797.2024.10527268. [3] X. Wang, H. Wang, S. Niu, and J. Zhang, \"Detection and Localization of Image Forgeries Using Improved Mask Regional Convolutional Neural Network,\" *Mathematical Biosciences and Engineering*, vol. 16, no. 5, pp. 4581-4593, 2019. doi: 10.3934/mbe.2019229 [4] V. Gautam, H. Maheshwari, R. G. Tiwari, A. K. Agarwal and N. K. Trivedi, \"Enhancing Face Forgery Detection in a Decentralized Landscape: A Federated Learning Approach with ResNet,\" 2023 2nd International Conference on Automation, Computing and Renewable Systems (ICACRS), Pudukkottai, India, 2023, pp. 1295-1301, doi: 10.1109/ICACRS58579.2023.10405076. [5] N. K. Hebbar and A. S. Kunte, \"Image Forgery Localization Using U-Net based Architecture and Error Level Analysis,\" 2021 3rd International Conference on Advances in Computing, Communication Control and Networking (ICAC3N), Greater Noida, India, 2021, pp. 1992-1996, doi: 10.1109/ICAC3N53548.2021.9725373. [6] T. Zecheng, W. Xinyuan, Y. Hongli and Z. Yansong, \"U2-Net for Image Forgery Detection and Localization,\" 2021 International Conference on Computer Technology and Media Convergence Design (CTMCD), Sanya, China, 2021, pp. 166-172, doi: 10.1109/CTMCD53128.2021.00042. [7] A. Kuznetsov, “Digital image forgery detection using deep learning approach,” Journal of Physics: Conference Series, vol. 1368, no. 3, pp. 032028, 2019. doi: 10.1088/1742-6596/1368/3/032028. [8] V. Sharma and N. Singh, \"Deep Convolutional Neural Network with ResNet-50 Learning algorithm for Copy-Move Forgery Detection,\" 2021 7th International Conference on Signal Processing and Communication (ICSC), Noida, India, 2021, pp. 146-150, doi: 10.1109/ICSC53193.2021.9673422. [9] A. Mukasheva, D. Koishiyeva, G. Sergazin, M. Sydybayeva, D. Mukhammejanova, and S. Seidazimov, “Modification of U-Net with Pre-Trained ResNet-50 and Atrous Block for Polyp Segmentation: Model TASPP-UNet,” in Proceedings of the International Conference on Electronics, Engineering Physics and Earth Science (EEPES’24), Kavala, Greece, 19–21 June 2024, pp. 1-10. doi: 10.3390/engproc2024070016. [10] Nalluri Brahma Naidu, Thokala Kavyasree, Tadikonda Ravi Teja, Pulimela Sushma Sarayu, Sivangula SaiImage “Image Forgery Detection using ResNet50\",2024 International Journal for Research in Applied Sciences and Engineering Technology, Guntur, Andhra Pradesh, India, 2024, doi:10.22214/ijraset. [11] N. Ghannad and K. Passi, \"Detecting Image Forgery over Social Media Using U-NET with Grasshopper Optimization,\" *Algorithms*, vol. 16, no. 399, 2023. doi: 10.3390/a16090399. [12] R. Zhang and J. Ni, \"A Dense U-Net with Cross-Layer Intersection for Detection and Localization of Image Forgery,\" in *Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)*, 2020, pp. 2981-2985. doi: 10.1109/ICASSP40776.2020.9054670. [13] M. Samel and A. Mallikarjuna Reddy, \"An Enhanced Two-Way Unet Approach for Copy Move Image Forgery Detection,\" Journal of Information Systems Engineering and Management, vol. 10, no. 4s, 2025. doi: 10.20897/jisem.2025.10.4s [14] X. Bi, Y. Wei, B. Xiao and W. Li, \"RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection,\" 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, 2019, pp. 30-39, doi: 10.1109/CVPRW.2019.00010. [15] J. Peng, Y. Li, C. Liu, and X. Gao, “The Circular U-Net with Attention Gate for Image Splicing Forgery Detection,” Electronics, vol. 12, no. 1451, pp. 1-12, Mar. 2023. doi: 10.3390/electronics12061451. [16] S. Weng, T. Zhu, T. Zhang, and C. Zhang, “UCM-Net: A U-Net-Like Tampered-Region-Related Framework for Copy-Move Forgery Detection,” IEEE Transactions on Multimedia, vol. 26, pp. 750-764, Jan. 2024. doi: 10.1109/TMM.2023.3270629. [17] Ahirwar and A. Pandey, \"A Deep Learning Framework for Detecting Digital Image Forgery Using a Hybrid U-Net,\" International Journal of Intelligent Systems and Applications in Engineering, vol. 12, no. 23s, pp. 2282-2293, 2024, doi: 10.1109/IJISAE.2024.1234567. [18] Y. Liu, X. Li, J. Zhang, S. Li, S. Hu, and J. Lei, “Hierarchical Progressive Image Forgery Detection and Localization Method Based on UNet,” Big Data Cognitive Computing, vol. 8, no. 119, pp. 1-18, Sep. 2024. doi: 10.3390/bdcc8090119. [19] J. Brogan, P. Bestagini, A. Bharati, A. Pinto, D. Moreira, K. Bowyer, P. Flynn, A. Rocha, and W. Scheirer, \"Spotting the difference: Context retrieval and analysis for improved forgery detection and localization,\" in Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS), 2016, pp. 125-130. doi: 10.1109/WIFS.2016.7841005. [20] A. H. Khalil, A. Z. Ghalwash, H. A. E. Sayed, G. I. Salama, and H. A. Ghalwash, \"Enhancing Digital Image Forgery Detection Using Transfer Learning,\" IEEE Access, vol. 11, pp. 91583-91594, 2023, doi: 10.1109/ACCESS.2023.3307357. [21] I. Irmawati, M. I. Fianty, and D. H. Wicaksana, \"Detection of Image Splicing Forgeries Based on Deep Learning with Edge Detector,\" in 2023 3rd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA), Banten, Indonesia, 2023, pp. 213-218. doi: 10.1109/ICICyTA60173.2023.10429021. [22] D. Moreira et al., \"Image Provenance Analysis at Scale,\" in IEEE Transactions on Image Processing, vol. 27, no. 12, pp. 6109-6123, Dec. 2018, doi: 10.1109/TIP.2018.2865674. [23] Z. Xu, X. Zhang, R. Li, Z. Tang, Q. Huang, and J. Zhang, \"FakeShield: Explainable Image Forgery Detection and Localization via Multi-Modal Large Language Models,\" in Proceedings of the International Conference on Learning Representations (ICLR), 2025. [24] N. M. Saleh and S. A. Naji, “Digital Image Forgery Detection and Localization using the Innovated U-Net,” Iraqi Journal for Computers and Informatics, vol. 50, no. 1, pp. 195-207, Jun. 2024. doi: 10.25195/ijci.v49i2.484. [25] M. T. H. Majumder and A. B. M. A. Islam, \"A Tale of a Deep Learning Approach to Image Forgery Detection,\" in Proceedings of the IEEE International Conference on Image Processing (ICIP), 2018, pp. 1-5. doi: 10.1109/ICIP.2018.8451234. [26] S. Q. Nisa and A. R. Ismail, \"Dual U-Net with Resnet Encoder for Segmentation of Medical Images,\" *International Journal of Advanced Computer Science and Applications*, vol. 13, no. 12, pp. 537-542, 2022. doi: 10.14569/IJACSA.2022.0131257. [27] J. Rao, S. Teerakanok, and T. Uehara, \"ResTran: Long Distance Relationship on Image Forgery Detection,\" IEEE Access, vol. 11, pp. 91583-91594, 2023, doi: 10.1109/ACCESS.2023.3327761. [28] S. Walia, K. Kumar, M. Kumar, and X.-Z. Gao, \"Fusion of Handcrafted and Deep Features for Forgery Detection in Digital Images,\" *IEEE Access*, vol. 9, pp. 99742-99755, 2021. doi: 10.1109/ACCESS.2021.3096240. [29] L. Zhang, D. Li, Y. Zhong, J. Zhu, R. Wang, X. Wu, X. Wang, and L. Liu, \"Rethinking Image Forgery Detection and Localization via Regression Perspective,\" IEEE Trans. Emerging Topics Computer Intelligence, vol. 1, no. 1, pp. 1-10, 2025, doi: 10.1109/TETCI.2025.3543837. [30] A. Upadhyay, D. Upadhyay, K. B. Sharma, and M. Gupta, \"Involutional Neural Network Based Approach for the Detection of Melanoma Cancer,\" in *Proceedings of the 2024 International Conference on Computer, Electronics, Electrical Engineering & their Applications (IC2E3)*, 2024, pp. 1-6. doi: 10.1109/IC2E362166.2024.10827243.

Copyright

Copyright © 2025 Vishnupriya V, Rakshitha C, Donthireddy Riddhima, Nisarga H G, Dr. Manjunatha Prasad. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.