AI Powered Drone Surveillance System for Real Time Object Detection in Unmanned Aerial Vehicles Using Deep Learning Approaches

Abstract

Deep learning algorithms, particularly CNNs, have significantly improved object detection accuracy in remote sensing applications. Unlike most UAV-based approaches, this project implements a static camera system utilizing CNN and YOLOv8 (a state-of-the-art one-stage detector) for real-time aerial image processing. The system is optimized for surveillance, environmental monitoring, and disaster response applications. While our current implementation uses fixed cameras, the architecture supports seamless UAV integration. This research examines the speed accuracy trade-offs between one-stage and two-stage detectors, demonstrating YOLOv8\'s ability to maintain both rapid inference and reliable detection performance. Experimental results validate the system\'s effectiveness in static deployments while its adaptability for dynamic drone-based scenarios.

Introduction

Drones (UAVs) have transformed industries like surveillance, rescue, mapping, and inspection through aerial imaging. However, real-time manual analysis of drone footage is inefficient. To address this, deep learning techniques, especially Convolutional Neural Networks (CNNs), are employed for tasks like object detection and anomaly recognition.

Yet, the risks associated with drones (e.g., safety, privacy, illegal use) are rising. Real-time drone detection is essential for ensuring public safety, particularly in sensitive zones like airports or prisons.

II. Literature Survey

Traditional drone detection techniques include:

• Radar (long-range but weak with small drones)

• Radio Frequency (RF) (effective but needs known frequencies)

• Acoustic sensors (cheap but sensitive to noise)

• Vision-based systems (accurate but dependent on lighting)

Recent research has shifted towards deep learning, particularly using:

• YOLO (You Only Look Once) for real-time detection

• Faster R-CNN, SSD, Mask R-CNN (less effective for small drones)

• Hybrid models like AVUHBO (uses both traditional features and deep learning)

Sensor fusion, combining radar, RF, and vision, significantly improves accuracy (up to 99%).

III. Methodology

The proposed system enhances YOLOv8 for real-time drone detection by:

• Modifying the architecture

• Tuning hyperparameters for drone-specific features

• Adapting to dynamic environments and optimizing data transmission

A. CNNs in Object Detection

CNNs extract hierarchical features from images:

1. Low-level: edges, textures

2. High-level: objects, shapes

CNNs predict both the class and location (bounding box) of objects in images.

B. YOLO Framework

• YOLO processes the entire image in one go.

• Divides image into a grid, predicts objects and locations per grid cell.

• Trained on labeled datasets with real-time detection capability.

IV. YOLOv8 Architecture

YOLOv8 is the latest and most advanced version of the YOLO series:

• Uses a single-pass CNN approach

• Detects objects at multiple scales

• Suitable for UAV footage and video frames

• Highly efficient in both speed and accuracy

V. Experimental Results

• Drone detection from UAVs and video streams was successful using YOLOv8.

• Precision and recall improved steadily during training.

• Combining datasets (e.g., USC + KCF-labeled) enhanced model performance compared to single-source training.

Conclusion

In this work we developed a real time AI powered drone surveillance system that integrates deep learning techniques for object detection in Unmanned Aerial Vehicles(UAV). The proposed YOLOV8 model processed process live stream feeds as well as pre-recorded videos to identify and track objects of interest. The proposed model uses drone-based data acquisition and intelligent analytics, offering scalable solutions for security, disaster management, and infrastructure inspection Deep learning (DL) is often considered a \"black-box\" solution for many problems, although ongoing research aims to address this perception. In the field of remote sensing, DL has already made significant contributions across various applications. Future directions focuses on usage of Transfer learning techniques involves leveraging pre-trained models on large-scale datasets to transfer learned knowledge from a source task to a target task. This approach conserves computational resources, accelerates training, and enhances model performance.

References

[1] Aytar, Y. (2014). Transfer learning for object category detection. PhD Thesis, 146. https://www.robots.ox.ac.uk/~vgg/publications/201 4/Aytar14a/aytar14a.pdf [2] Chamarty, A. (2020). Fine-Tuning of Learning Rate for Improvement of Object Detection Accuracy. Proceedings - 2020 IEEE India Council International Subsections Conference, INDISCON 2020, 135–141. https://doi.org/10.1109/INDISCON50162.2020.000 38 [3] Cui, X., Zhang, W., Tüske, Z., & Picheny, M. (2018). Evolutionary stochastic gradient descent for optimization of deep neural networks. Advances in Neural Information Processing Systems, 2018- Decem (NeurIPS), 6048–6058. [4] Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 580–587. https://doi.org/10.1109/CVPR.2014.81 [5] Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. 32nd International Conference on Machine Learning, ICML 2015, 1, 448–456. [6] Koo, K. M., & Cha, E. Y. (2017). Image recognition performance enhancements using image normalization. Human-Centric Computing and Information Sciences, 7(1). https://doi.org/10.1186/s13673-017-0114-5 [7] Lin, T. Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020). Focal Loss for Dense Object Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2), 318–327. https://doi.org/10.1109/TPAMI.2018.2858826 [8] Liu, W., Anguelov, D., Erhan, D., & Fu, C.-Y. (2016). SSD:Single Shot MultiBox Detector. European Conference on Computer Vision, 1(January 2017), 398–413. https://doi.org/10.1007/978-3-319-46448-0_2 [9] MuhammadHussain, YOLO-v1 to YOLO-v8, the Rise of YOLO and Its Complementary Nature toward Digital Manufacturing and Industrial Defect Detection. https://doi.org/10.3390/machines11070677.

Copyright

Copyright © 2025 Pulagam Naveen Kumar, Rampranay Ponnamaneni, Thalla Tirumaleswarlu, B. Rajesh. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.