Utilizing Robotic System for Disaster Scene Classification - A Deep Learning Approach

Abstract

Disaster scene classification plays a vital role in emergency management by facilitating rapid assessment and response to scenarios such as floods, earthquakes, and wildfires. Traditional image classification methods face challenges due to the complexity and variability of disaster scenes, which often include irregular patterns and diverse environmental factors. In recent years, deep learning, particularly convolutional neural networks (CNNs), has demonstrated significant potential in improving the accuracy of disaster scene classification. This project integrates a CNN-based approach with a remote-controlled robot equipped with real-time image-capturing capabilities. The robot navigates disaster zones, capturing images that are processed using CNN architecture like VGG16 and VGG19 to classify disaster scenes efficiently. The robotic system enhances situational awareness by autonomously collecting vital information in hazardous environments, transmitting real-time data for classification, and providing timely insights for emergency response. This integration of robotics with deep learning not only automates disaster scene classification but also reduces reliance on large, labeled datasets, improving performance and response effectiveness.

Introduction

Overview:
Natural disasters like floods, earthquakes, and wildfires pose serious threats to life and property. Traditional disaster response methods face delays and safety risks. This project introduces a robotic system integrated with deep learning—especially Convolutional Neural Networks (CNNs) such as VGG16 and VGG19—to improve real-time disaster scene classification and emergency response.

Key Components:

1. Robotic System:

• Equipped with cameras, sensors, and wireless communication modules.

• Navigates disaster-affected areas autonomously or remotely.

• Captures real-time images and environmental data.

• Sends images to a server for classification using trained CNNs.

2. Deep Learning for Scene Classification:

• CNN models (VGG16, VGG19) identify disaster types such as floods, fires, cyclones, and collapsed structures.

• Enables fast decision-making by first responders.

• System architecture includes image preprocessing, training, validation, and testing datasets.

Literature Review Highlights:

• Deep learning improves disaster scene analysis, especially when integrated into mobile robots and remote sensing platforms.

• CNNs, transfer learning, and ensemble models like VRBaggedNet show high accuracy in disaster classification tasks.

• Applications include forest fire detection, landslide monitoring, 3D mapping, and victim localization.

• Studies show the flexibility of AI models to adapt to various disaster scenarios with high-resolution imagery and mobile platforms.

Proposed System Contributions:

• Real-time image classification using CNNs on embedded systems (e.g., Raspberry Pi, ESP32).

• Wireless communication ensures remote monitoring via cloud servers.

• Combines AI, robotics, and sensors for enhanced situational awareness and emergency planning.

Hardware & Software Used:

• Software: Python, TensorFlow, OpenCV, Flask, MQTT, Google Colab, TensorFlow Lite.

• Hardware: Raspberry Pi, ESP32, high-resolution cameras, DC motors, gas sensors, IMU, and Li-Po batteries.

• Communication: Wi-Fi, MQTT/WebSockets for real-time data transfer.

Challenges in Existing Systems:

• Dependence on large labeled datasets.

• Limited wireless coverage and processing power in field conditions.

• Need for better real-time inference and environmental adaptability.

Conclusion

The use of deep learning techniques in disaster management has been investigated in this research study, with particular attention paid to robot integration, domain- specific applications, and scene categorization. After a thoroughanalysisofpertinentresearch,deeplearning-more specifically, CNNs has become an effective method for categorizing catastrophe scenes, enablingaccurate andrapid identification of various natural disasters such as cyclones, earthquakes, floods, and wildfires. Additionally, domain- specific applications have demonstrated the versatility of deep learning in addressing specific challenges within disaster management, including remote sensing imagery analysis, social media sensing, and forest fire detection. Furthermore, the integration of robots equipped with advanced sensors and cameras offers a promising approach to enhance situational awareness and streamline response efforts in disaster scenarios. By leveraging deep learning algorithms for real-time data analysis and decision-making, these robotic platforms can autonomously navigate through affected areas, collect vital information, and communicate with emergency responders, thereby mitigating risks and improving overall response effectiveness.

References

[1] Xu X. Chen, Y. Zhang, J, Chen, Y. Anandhan, P. Manickam, A. “,A novel approach for scene classification from remote sensing images using deep learning methods,”, European Journal of Remote Sensing, Vol. 54, no. 2, pp. 383-395, 2021. [2] Dotel, S. Shrestha, A. Bhusal, A. Pathak, R. Shakya, A. Panday, S. P. “Disaster assessment from satellite imagery by analysing topographical features using deep learning” In Proceedings of the 2020 2nd International Conference on Image, Video and Signal Processing ,pp. 86-92, March 2020. [3] Gao, Y. Gao, L. L, X. “A generative adversarial network based deep learning method for low-quality defect image reconstruction and recognition”, IEEE Transactions on Industrial Informatics, vol.17, no.5, 3231- 3240, 2020. [4] Tang S., Chen Z. “Understanding natural disaster scenes from mobile images using deep learning”, Applied Sciences, vol. 11, no. 9, pp. 3952, 2021 [5] Arnold S., Yamazaki K. “Real-time scene parsing by means of a convolutional neural network for mobile robots in disaster scenario”, In 2017 IEEE International conference on information and automation (ICIA) ,pp. 201-207. IEEE, July 2017. [6] Ghosh A., et al. . ”Disaster Scene Classification using Deep Learning with Contextual Information.” IEEE Transactions on Geo- science and Remote Sensing, 2023. [7] Bird J. J., Faria, D. R., Eka´rt, A., Ayrosa P. P,” From simulation to reality: CNN transfer learning for scene classification.” In 2020 IEEE 10th International Conference on Intelligent Systems (IS) ,pp. 619625, IEEE, 2020. [8] Li, L., Ota, K., Dong, M., Borjigin, W. “Eyes in the dark: Distributed scene understanding for disaster management. IEEE transactions on parallel and distributed systems, vol. 28, no. 12, pp. 3458-3471, 2017. [9] Alawad, Wedad, Nadhir Ben Halima, and Layla Aziz. \"An unmanned aerial vehicle (uav) system for disaster and crisis management in smart cities.\" Electronics 12, no. 4,p. 1051,2023 [10] Khan, A., Hassan, B., Khan, S., Ahmed, R., Abuassba, A. “DeepFire: A novel dataset and deep transfer learning benchmark for forest fire detection”, Mobile Information Systems, 2022. [11] Lu, H., Ma, L., Fu, X., Liu, C., Wang, Z., Tang, M., Li, N. “Landslides information extraction using object-oriented image analysis paradigm based on deep learning and transfer learning”, Remote Sensing, vol. 12, no. 5, p. 752, 2020. [12] Zhang, Y., Zong, R., Wang, D. “A hybrid transfer learning approach to migratable disaster assessment in social media sensing” In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM),pp. 131-138, IEEE, 2020 [13] Zhang, Z., Guo, H., Nejat, G., Huang, P. “Finding disaster victims: A sensory system for robot-assisted 3D mapping of urban”, In Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 3889-3894. IEEE, 2007 [14] Hanif M, Tahir, M. A. Rafi, “, VRBagged-Net: Ensemble Based Deep Learning Model for Disaster Event Classification. Electron- ics”, vol.10, no.12, 1411,2021. [15] Gopika D., Sivadasan H., Jiresh, P., Sucheta G. S., Patta S. Y. “Human Detection Robot For Disaster Management”. International Journal of Engineering Applied Sciences and Technology, vol. 5, no. 10, pp. 193-199, 2021. [16] Krishna, A. R., Bala, G. S., Chakravarthy, A. S. N., Sarma, B. B. P., Alla, G. S, “Design of a rescue robot assist at fire disaster”, International Journal of Computer Applications, vol. 975, p. 888, 2012 [17] Doroodgar, B. Liu, Y. Nejat, G. “A learning-based semi- autonomous controller for robotic exploration of unknown disaster scenes while searching for victims”, IEEE Transactions on Cybernetics,vol.44,no.12,pp. 2719-2732,2014. [18] Petrovska, B. Zdravevski, E., Lameski, P. Corizzo, R., S?tajduhar, I., Lerga, J. “Deep learning for feature extraction in remote sensing: A case-study of aerial scene classification”, Sensors, vol. 20, no. 14, p. 3906, 2020.

Copyright

Copyright © 2025 Mosmi G. Belsare , Prajjwalsingh S. Rajput, Disha S. Patil, Trupti S. Raghatwan, Prof. A. M. Karanjkar . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.