Object Detection and Tracking, Searching Using Computer Vision and YOLO Technique

Abstract

Real-time visual understanding has become a cornerstone of modern intelligent systems, spanning surveillance, autonomous navigation, retail analytics, and smart-city infrastructure. The system architecture is described in detail, including data-flow diagrams, use-case and sequence diagrams, an activity diagram, and a relational database design for persisting detection and tracking metadata. An experimental evaluation performed on standard benchmark-style data demonstrates that the proposed pipeline attains a mean Average Precision (mAP@0.5) of approximately 0.91, a tracking identity-switch rate reduced by 38% relative to a naive frame-by-frame detector, and a sustained throughput of 61 frames per second on a mid-range GPU, outperforming several baseline architectures compared in this study. The results confirm that combining YOLOv8 with Deep SORT and a dedicated search layer yields a practical, extensible platform for real-time object detection, tracking, and retrieval applications.

Introduction

The rapid advancement of digital imaging, high-resolution cameras, and affordable computing has accelerated the development of automated visual analysis systems for applications such as surveillance, traffic monitoring, autonomous vehicles, warehouse automation, and human–computer interaction. These applications rely on three core computer vision tasks: object detection, multi-object tracking, and object searching. Traditional object detection methods based on handcrafted features such as Haar Cascades, HOG, and SIFT suffered from limited robustness, while two-stage deep learning detectors like R-CNN and Faster R-CNN achieved higher accuracy but were computationally expensive for real-time applications.

This study proposes an integrated computer vision system that combines YOLOv8 for real-time object detection, Deep SORT for multi-object tracking, and an object searching module into a unified platform capable of processing static images, prerecorded videos, and live camera streams. The system detects objects, tracks them across frames using persistent identities, stores detection and tracking information in a relational database, and enables users to search objects by class, visual attributes, or tracking ID.

The literature review highlights the evolution of object detection from two-stage region proposal methods to modern deep learning techniques, while emphasizing the continued importance of computer vision preprocessing using OpenCV. Existing systems are limited by manual surveillance, high computational latency, poor tracking under occlusion, fragmented object identities, and limited search capabilities that often rely only on metadata rather than visual content.

To overcome these limitations, the proposed system employs the single-stage, anchor-free YOLOv8 detector, which provides fast and accurate object detection, and Deep SORT, which combines Kalman filter-based motion prediction with appearance-based re-identification to maintain consistent object identities during occlusions and crossing trajectories. The computer vision pipeline includes frame acquisition, resizing, normalization, optional image enhancement, and adaptive frame skipping to maintain real-time performance.

The object searching module supports class-based, attribute-based (e.g., color), and identity-based searches, allowing efficient retrieval of detected objects and their trajectories. Detection results, confidence scores, timestamps, and tracking information are stored in a database, enabling reporting, analytics, and historical searches.

The system is implemented in Python using PyTorch for YOLOv8, OpenCV for image and video processing, and SQLite (portable to PostgreSQL/MySQL) for data storage. Functional modules include image detection, video detection, live camera detection, object tracking, object searching, and report generation. Reports summarize detection statistics and can be exported in formats such as CSV and PDF.

Experimental evaluation uses metrics including precision, recall, mean Average Precision (mAP), frames per second (FPS), and identity-switch rate. The results demonstrate that the proposed YOLOv8–Deep SORT framework provides accurate, real-time object detection and tracking, robust identity preservation under occlusion, efficient visual object retrieval, and an integrated solution suitable for intelligent surveillance and other computer vision applications.

Conclusion

This paper presented an integrated system for object detection, multi-object tracking, and object searching built around the YOLOv8 detection architecture, the Deep SORT tracking algorithm, and a dedicated computer-vision-driven search module. The system supports static images, recorded video files, and live camera streams within a unified pipeline, persists detection and tracking metadata in a structured relational database, and exposes class-based, attribute-based, and identity-based search capabilities alongside a reporting dashboard. Experimental evaluation demonstrated that the proposed pipeline achieves a mean Average Precision of approximately 0.91, sustains a throughput of 61 frames per second, and reduces tracking identity switches by 38% relative to a motion-only baseline, while search queries resolve with sub-100-millisecond latency against a moderately sized metadata index. These results confirm that combining a fast, accurate single-stage detector with an appearance-aware tracker and an integrated search layer yields a practical and extensible foundation for real-time video analytics applications across surveillance, traffic management, retail, industrial, and environmental-monitoring domains. Future work will focus on multi-camera re-identification, edge deployment, and natural-language search interfaces to further broaden the applicability of the system.

References

[1] R. Girshick, J. Donahue, T. Darrell, and J. Malik, \"Rich feature hierarchies for accurate object detection and semantic segmentation,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 580-587. [2] R. Girshick, \"Fast R-CNN,\" in Proc. IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, 2015, pp. 1440-1448. [3] S. Ren, K. He, R. Girshick, and J. Sun, \"Faster R-CNN: Towards real-time object detection with region proposal networks,\" IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 6, pp. 1137-1149, Jun. 2017. [4] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, \"You only look once: Unified, real-time object detection,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 779-788. [5] J. Redmon and A. Farhadi, \"YOLO9000: Better, faster, stronger,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6517-6525. [6] J. Redmon and A. Farhadi, \"YOLOv3: An incremental improvement,\" arXiv preprint arXiv:1804.02767, 2018. [7] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, \"YOLOv4: Optimal speed and accuracy of object detection,\" arXiv preprint arXiv:2004.10934, 2020. [8] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, \"YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,\" in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, 2023, pp. 7464-7475. [9] G. Jocher, A. Chaurasia, and J. Qiu, \"Ultralytics YOLOv8,\" Ultralytics, 2023. [Online]. Available: https://github.com/ultralytics/ultralytics [10] W. Liu et al., \"SSD: Single shot multibox detector,\" in Proc. European Conf. Computer Vision (ECCV), Amsterdam, Netherlands, 2016, pp. 21-37. [11] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollar, \"Focal loss for dense object detection,\" in Proc. IEEE Int. Conf. Computer Vision (ICCV), Venice, Italy, 2017, pp. 2980-2988. [12] T.-Y. Lin et al., \"Feature pyramid networks for object detection,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 2117-2125. [13] N. Carion et al., \"End-to-end object detection with transformers,\" in Proc. European Conf. Computer Vision (ECCV), Glasgow, UK, 2020, pp. 213-229. [14] K. He, G. Gkioxari, P. Dollar, and R. Girshick, \"Mask R-CNN,\" in Proc. IEEE Int. Conf. Computer Vision (ICCV), Venice, Italy, 2017, pp. 2961-2969. [15] A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, \"Simple online and realtime tracking,\" in Proc. IEEE Int. Conf. Image Processing (ICIP), Phoenix, AZ, USA, 2016, pp. 3464-3468. [16] N. Wojke, A. Bewley, and D. Paulus, \"Simple online and realtime tracking with a deep association metric,\" in Proc. IEEE Int. Conf. Image Processing (ICIP), Beijing, China, 2017, pp. 3645-3649. [17] Y. Zhang et al., \"ByteTrack: Multi-object tracking by associating every detection box,\" in Proc. European Conf. Computer Vision (ECCV), Tel Aviv, Israel, 2022, pp. 1-21. [18] R. E. Kalman, \"A new approach to linear filtering and prediction problems,\" Trans. ASME J. Basic Eng., vol. 82, no. 1, pp. 35-45, 1960. [19] H. W. Kuhn, \"The Hungarian method for the assignment problem,\" Naval Research Logistics Quarterly, vol. 2, no. 1-2, pp. 83-97, 1955. [20] N. Dalal and B. Triggs, \"Histograms of oriented gradients for human detection,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, 2005, pp. 886-893. [21] D. G. Lowe, \"Distinctive image features from scale-invariant keypoints,\" Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004. [22] P. Viola and M. Jones, \"Rapid object detection using a boosted cascade of simple features,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Kauai, HI, USA, 2001, pp. 511-518. [23] O. Barnich and M. Van Droogenbroeck, \"ViBe: A universal background subtraction algorithm for video sequences,\" IEEE Trans. Image Process., vol. 20, no. 6, pp. 1709-1724, Jun. 2011. [24] G. Bradski, \"The OpenCV library,\" Dr. Dobb\'s Journal of Software Tools, vol. 25, no. 11, pp. 120-125, 2000. [25] A. Krizhevsky, I. Sutskever, and G. E. Hinton, \"ImageNet classification with deep convolutional neural networks,\" in Proc. Advances in Neural Information Processing Systems (NeurIPS), Lake Tahoe, NV, USA, 2012, pp. 1097-1105. [26] K. He, X. Zhang, S. Ren, and J. Sun, \"Deep residual learning for image recognition,\" in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. [27] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, \"The PASCAL Visual Object Classes (VOC) challenge,\" Int. J. Comput. Vis., vol. 88, no. 2, pp. 303-338, 2010. [28] T.-Y. Lin et al., \"Microsoft COCO: Common objects in context,\" in Proc. European Conf. Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 740-755. [29] S. Ioffe and C. Szegedy, \"Batch normalization: Accelerating deep network training by reducing internal covariate shift,\" in Proc. Int. Conf. Machine Learning (ICML), Lille, France, 2015, pp. 448-456. [30] D. P. Kingma and J. Ba, \"Adam: A method for stochastic optimization,\" in Proc. Int. Conf. Learning Representations (ICLR), San Diego, CA, USA, 2015, pp. 1-15

Copyright

Copyright © 2026 Banka Vinay, Mrs. Pravitha R. Prasad. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.