Counterfeit Medicine Recognition System

Abstract

Counterfeit and misbranded drugs remain a major challenge in contemporary healthcare, threatening patient safety and treatment efficacy. To overcome this critical challenge, a hybrid system is constructed that combines Optical Character Recognition (OCR), Scale-Invariant Feature Transform (SIFT) algorithm, and state-of-the-art preprocessing methods for automated drug identification and authentication. This system processes both text and image data from medicine packs, making it extremely adaptable to different packaging types and real-world image differences. The underlying novelty is the combination of OCR-based data extraction with SIFT-based visual feature analysis, enhancing robustness even with inhomogeneous lighting, orientation, and background conditions. The two-modality solution increases accuracy in medicine verification but at real-time capability. It offers fast and accurate verification, is multilingual text recognition compatible, and fits any format from tablets to injectables and syrups. Performance is assessed using critical parameters such as F1-score, recall, precision, and accuracy over a customized dataset with reliable consistency across various input scenarios. By being a scalable, portable, and efficient solution, the system described here has a high potential for adoption in pharmacies, hospitals, and rural medical centers and has the capability to increase drug traceability and patient safety in day-to-day medical practice.

Introduction

The proliferation of fake and mislabeled drugs poses serious public health risks, including incorrect treatments, harmful interactions, and even death. Accurate real-time authentication of medicines at the point of care is critical.

Proposed Solution:
Combining computer vision and artificial intelligence (AI)—specifically object detection (YOLOv4) and Optical Character Recognition (OCR)—can automate drug identification and label verification. OCR is used to extract text from medicine packaging and handwritten prescriptions, improving accuracy and reducing manual errors. Machine learning algorithms like K-Nearest Neighbors (KNN) enhance classification and verification when integrated with OCR outputs.

Literature Insights:

• AI, machine learning, and computer vision have improved prescription analysis, drug recommendation, and error detection.

• OCR with Tesseract, combined with preprocessing techniques, enables accurate text extraction from scanned documents.

• Feature-based image analysis (SIFT, ORB, FLANN, RANSAC) allows robust matching of medicine images despite noise, occlusion, or scale variations.

• Previous systems mostly relied on either OCR (text) or visual feature matching, limiting performance in real-world settings with complex layouts and handwritten content.

Proposed PIRCS Framework:

• Multimodal Approach: Integrates OCR-based text extraction with feature-based image matching (SIFT + ORB) in a single pipeline for robust medicine verification.

• Pipeline: Image acquisition → Preprocessing → OCR extraction → Feature extraction → Feature matching → Final medicine identification.

• Data Used: Visual images of medicines under varying conditions and structured textual data (name, ingredients, dosage, substitutes).

• Algorithms:

  • SIFT: Detects distinctive, scale- and rotation-invariant features in medicine images.

  • RANSAC: Eliminates outliers during keypoint matching to improve reliability.

  • FLANN: Matches features efficiently between input and reference images.

Operational Flow:

1. Capture or upload medicine image.

2. Preprocess (grayscale, thresholding, noise removal).

3. Apply OCR to extract name, manufacturing/expiry dates, batch number.

4. Detect image keypoints with SIFT and match against reference database using FLANN.

5. Verify medicine identity based on match score.

6. Log image, extracted data, and verification status.

Results:

• Implementation using Python, OpenCV, Tesseract OCR, and SIFT on a controlled dataset showed accurate verification and classification across varied packaging conditions.

• The multimodal integration ensures reliable detection, authentication, and real-time classification, suitable for practical healthcare settings, including rural or mobile environments.

Key Takeaways:

• The PIRCS system addresses limitations of single-modality approaches by combining text and image analysis.

• Robust preprocessing, hybrid feature descriptors, and OCR enhance performance on noisy, low-resolution, or complex prescription images.

• Supports real-time, edge-device deployment for medicine verification, improving safety and reducing errors.

Conclusion

Recent research establishes the growing risk of counterfeiting and misbranding drugs, which completely detracts from patient safety as well as from the efficacy of drugs. Using OCR to extract image-based text and SIFT for visual keypoint matching, we developed an intelligent system capable of successfully identifying drugs using image processing techniques. The innovation of the solution is its hybridity—textual and image recognition merged together for better robustness, ability to detect manipulated packaging by keypoint matching, and its modularity for easy implementation in healthcare systems or mobile applications. All these make it stronger, more flexible, and scalable verification. The advantages of the system are fast detection, support for multiformat packaging, and high accuracy in low image quality. Its main limitation lies in its dependency on high-quality images and limited support for printed text on parcels. Future enhancements may include the integration of deep learning models and improved mobile compatibility to facilitate real-time verification and enhance accessibility.

References

[1] Caliolio, Marielle Hannah, Joshua Frias, and Charmaine Paglinawan. \"Integration of a smart medicine container with medicine sorting using YOLOv4 and OCR.\" 2022 13th International Conference on Computing Communication and Networking Technologies (ICCCNT). IEEE, 2022.” [2] ?Kavin, S., and C. P. Shirley. \"OCR-Based Extraction of Expiry Dates and Batch Numbers in Medicine Packaging for Error-Free Data Entry.\" 2024 7th International Conference on Circuit Power and Computing Technologies (ICCPCT). Vol. 1. IEEE, 2024. [3] Sanjeevaiah, Mr K., et al. \"Medical Prescription Optical Character Recognition.\" [4] ?Kumar, Anjani, et al. \"OCR based medical prescription and report analyzer.\" AIP Conference Proceedings. Vol. 2424. No. 1. AIP Publishing, 2022. [5] Kathpalia, Nishant, Gabriel Nixon Raj, and Madhavan Venkatesh. \"A Smart Healthcare Companion with Tesseract OCR and KNN Integration.\" Technologies for Energy, Agriculture, and Healthcare. CRC Press, 2025. 284-294. [6] Agrawal, Mayur, et al. \"A review of artificial intelligence in medical prescription analysis.\" 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN). IEEE, 2023. [7] Flores, Christopher A., and Rodrigo Verschae. \"Combining Regular Expressions and Supervised Algorithms for Clinical Text Classification.\" International Conference on Intelligent Data Engineering and Automated Learning. Cham: Springer Nature Switzerland, 2023. [8] Ponnuru, Mahesh, and A. Likhitha. \"Image-Based Extraction of Prescription Information using OCR-Tesseract.\" Procedia Computer Science 235 (2024): 1077-1086. [9] Burger, Wilhelm, and Mark J. Burge. \"Scale-invariant feature transform (SIFT).\" Digital Image Processing: An Algorithmic Introduction. Cham: Springer International Publishing, 2022. 709-763. [10] Bellavia, Fabio. \"SIFT matching by context exposed.\" IEEE transactions on pattern analysis and machine intelligence 45.2 (2022): 2445-2457. [11] Gupta, Megha, and Priyanka Singh. \"An image forensic technique based on SIFT descriptors and FLANN based matching.\" 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT). IEEE, 2021. [12] Bellavia, Fabio. \"SIFT matching by context exposed.\" IEEE transactions on pattern analysis and machine intelligence 45.2 (2022): 2445-2457. [13] Chhabra, Payal, Naresh Kumar Garg, and Munish Kumar. \"Content-based image retrieval system using ORB and SIFT features.\" Neural Computing and Applications 32.7 (2020): 2725-2733. [14] Wang, Shigang, Zhenjin Guo, and Yang Liu. \"An image matching method based on sift feature extraction and FLANN search algorithm improvement.\" Journal of Physics: Conference Series. Vol. 2037. No. 1. IOP Publishing, 2021. [15] Barath, Daniel, Luca Cavalli, and Marc Pollefeys. \"Learning to find good models in RANSAC.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.

Copyright

Copyright © 2025 Prof. Vijay Mane, Siddhika Bhagat, Swarali Mahajan, Prachi Shrirao. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.