AI-Based Intelligent HSRP Compliance Screening and Decision Support System Using Computer Vision

Abstract

HSRPs are designed to make it uniform and to facilitate quick identification of vehicles but the surveillance of vast amounts of traffic manually is expensive and unfeasible. In this report, an AI-powered HSRP compliance screening and decision-support system is proposed that focuses on traffic-video screening and is based on computer vision. The infrastructure uses a YOLO-based traffic video analysis module for vehicle and HSRP detection, and Byte Track for vehicle tracking to associate vehicles across frames. Vehicle detections are later improved through best-frame selection, and multiple observations are filtered to accumulate evidence on each vehicle before OCR recognition. PaddleOCR is selected as the main OCR engine, and OCR results are fused across frames using temporal evidence while considering aspect ratio and registration-format constraints to reduce the impact of misclassification. The visible evidence of HSRPs, including IND-band evidence, plate quality, aspect ratio, and OCR confidence, is combined into a screening assessment. Tracking, OCR, HSRP screening, explainable prediction and reasoning, and human image-rejection and failure mode checks are combined within a streamlined vehicle-screening tool, distinguishing vehicles that are likely to be compliant from those that are potentially subject to further human review. All plate and video evidence, and screening information, reasoning, and failure modes are presented via an API-powered display available for download and human review. The entire system is validated through end-to-end traffic video analysis and automated testing, though it does not seek to achieve comprehensive legal authentication of HSRP security features.

Introduction

The AI-Based Intelligent HSRP Compliance Screening and Decision Support System is designed to improve vehicle number plate recognition and High-Security Registration Plate (HSRP) screening using artificial intelligence and computer vision. Traditional Automatic Number Plate Recognition (ANPR) systems may face difficulties due to poor lighting, motion blur, low image resolution, vehicle movement, and partially obscured plates.

The proposed system uses a multi-stage processing pipeline. A YOLO-based object detector identifies vehicles, while ByteTrack tracks them across consecutive video frames. A license plate detector locates number plates and associates them with the corresponding vehicles. Plate-quality analysis and best-frame selection identify clearer images for recognition. PaddleOCR is then used to extract registration numbers, while temporal OCR analysis combines results from multiple frames to improve recognition reliability.

After recognizing the number plate, the system analyses visible HSRP-related features, including plate quality, OCR confidence, consistency across frames, registration-format validity, plate aspect ratio, and visible Indian IND-band evidence. Based on the available evidence, the system generates one of two outcomes: Likely Compliant or Needs Manual Verification.

The results are stored in an SQLite database and displayed through a FastAPI dashboard. The dashboard allows human operators to review the evidence, understand the reasons behind a screening result, correct OCR output when necessary, and conduct manual verification.

The literature survey examines existing research on vehicle detection, number plate recognition, object tracking, OCR, HSRP feature analysis, and explainable artificial intelligence. It identifies a research gap in combining these techniques into a unified system that uses evidence from multiple video frames and provides transparent explanations for its decisions.

The main contribution of the proposed system is the integration of vehicle detection, tracking, plate recognition, temporal evidence analysis, visible HSRP screening, database storage, and explainable human-supervised decision support within a single framework. Rather than automatically declaring a plate authentic or imposing enforcement actions, the system assists authorities by identifying vehicles that appear compliant and those requiring further inspection. This approach aims to improve screening reliability, transparency, and efficiency while retaining human oversight for final verification.

Conclusion

This work presented an AI-Based Intelligent HSRP Compliance Screening and Decision Support System using Computer Vision for traffic-video analysis. The developed prototype integrates vehicle detection, multi-object tracking, license plate detection, plate-to-vehicle association, best-frame selection, OCR, temporal validation, visible HSRP screening, database storage, explainability, and human verification within a single workflow. The use of ByteTrack allows the same vehicle to be followed across multiple frames, enabling the system to collect repeated plate observations instead of relying on a single image. Best-frame selection improves the quality of the evidence used for OCR, while temporal OCR analysis helps identify more stable registration candidates by comparing observations over time. The system also adds to traditional ANPR by estimating visibly viewable HSRP related evidence like plate quality, OCR accuracy, registration format plausibility, plate aspect ratio, time consistency and IND-band evidence. From these, the system outputs Likely Compliant or Needs Manual Verification decisions. The system uses this conservative approach to avoid turning any unknowns into legal violations, instead of handing off these to a human operator for review. The implemented prototype was validated using automated tests and end-to-end traffic-video processing. A verified run processed 60 video frames, maintained 21 vehicle tracks, generated 14 best-plate updates, and performed 13 OCR executions. In addition, 52 out of 52 automated tests passed, confirming the functional integration of the major system modules. The FastAPI-based dashboard and SQLite storage layer provide a practical interface for reviewing processed vehicles, OCR results, screening decisions, and supporting evidence. The “Why This Result?” functionality improves transparency by showing the evidence and warning factors that influenced each screening result, while manual correction and human review preserve human control over uncertain decisions. The system is not designed to supplant HSRP authentication or law-enforcement processing. Some physical security attributes, including hologram authentication, laser-etched ID numbers, manufacturer information, and anti-tampering features, are difficult to confirm from plain-vanilla surveillance footage. Consequently, the system should be employed as an AI-assisted screening and decision support tool.

References

[1] R. Al-Batat, A. Angelopoulou, S. Premkumar, J. Hemanth, and E. Kapetanios, “An End-to-End Automated License Plate Recognition System Using YOLO Based Vehicle and License Plate Detection with Vehicle Classification,” Sensors, vol. 22, no. 23, Art. no. 9477, 2022, doi: 10.3390/s22239477. [2] R. Adak, A. Kumbhar, R. Pathare, and S. Gowda, “Automatic Number Plate Recognition (ANPR) with YOLOv3-CNN,” arXiv preprint arXiv:2211.05229, 2022. [3] S. Dhyani and V. Kumar, “Real-Time License Plate Detection and Recognition System using YOLOv7x and EasyOCR,” in Proc. 2023 Global Conference on Information Technologies and Communications (GCITC), Bangalore, India, Dec. 2023, doi: 10.1109/GCITC60406.2023.10425814. [4] L. You, Y. Chen, C. Xiao, C. Sun, and R. Li, “Multi-Object Vehicle Detection and Tracking Algorithm Based on Improved YOLOv8 and ByteTrack,” Electronics, vol. 13, no. 15, Art. no. 3033, 2024, doi: 10.3390/electronics13153033. [5] A. Y. Sugiyono, K. Adrio, K. Tanuwijaya, and K. M. Suryaningrum, “Extracting Information from Vehicle Registration Plate using OCR Tesseract,” Procedia Computer Science, vol. 227, pp. 932–938, 2023, doi: 10.1016/j.procs.2023.10.600. [6] H. K. Gianey and A. Goyal, “Localization and Recognition of High Security Registration Plate for Indian Vehicle,” International Journal of Computer & Organization Trends, vol. 4, no. 2, pp. 16–19, 2014, doi: 10.14445/22492593/IJCOT-V6P304. [7] G. Badamikar, G. Parashetti, V. Handiganur, P. Singh, and M. M. Raikar, “High Security Registration Plate Detection and Classification Using Federated Learning,” Procedia Computer Science, vol. 260, pp. 1129–1136, 2025, doi: 10.1016/j.procs.2025.03.298. [8] A. Kuznietsov, B. Gyevnar, C. Wang, S. Peters, and S. V. Albrecht, “Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 12, pp. 19342–19364, Dec. 2024, doi: 10.1109/TITS.2024.3474469.

Copyright

Copyright © 2026 Madhur Madankar, Karan Shahu, Nikhil Netam, Divyansh Verma, Vedant Mankar, Yash Karme, Sumalatha Bandari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.