Authors: Shilpasree S
DOI Link: https://doi.org/10.22214/ijraset.2022.47751
Certificate: View Certificate
Owing to the exponentially growing amount of transportation on the road, the number of mishaps on daily basis is also growing at a shocking rate. There is one death every 4 minutes due to road mishaps in India. One of the foremost causes of these road mishaps to occur is the poor road environment. In the present work, Reinforcement learning is used to classify road quality. Reinforcement learning is a part of machine learning. Suitable actions are taken to maximize the reward. Reinforcement learning is different from supervised learning, in reinforcement learning, there is no trained dataset but the agent learns from its experience and decides what actions should be performed for the given task in order to achieve the maximum reward. We use Q-learning a Reinforcement learning algorithm that will find the best action, in the given current state of an environment. It chooses the action randomly and aims to maximize the reward. An environment is created in which policy, action, and rewards to classify road structure quality are defined. The dataset has 15 features and 5 classes based on which policy is defined that defines the next action to be taken based on the state. Training and Testing are done with 80:20 ratios. Accuracy for q-learning is 93.62% and enhanced q-learning is 93.74% obtained. A confusion matrix with precision, F1 score, recall, and support is calculated. The accuracy obtained for different training and testing model ratios 60:40, 70:30, 80:20, and 90:10 is 93.19, 91.06, 91.51, and 91.86 respectively. The average accuracy for the training and testing model is 91.86.
Owing to the exponentially growing amount of transportation on the road, the number of mishaps on daily basis is also growing at a shocking rate. There is one death every 4 minutes due to road mishaps in India. One of the foremost causes of these road mishaps to occur is the poor road environment. The geometry of the road shows a major part in road crash incidences as well as the crash ruthlessness level. Diverse features of road plans are essential.
A. Road Factors Affecting Road Welfare
The system architecture uses the features that need to be considered from requirements and provides a skeleton of the system that needs to be prepared for execution. The system contains a road environment that is distributed as accident zones and junction roads. An agent will be placed in an environment, with the initial state and random actions will be performed by the agent to reach the destination. If there are any hotspots such as drainage pavement, anti-slip pavement, pedestrian crossing zone, high traffic zone,
C. Reinforcement learning
Reinforcement learning is a machine learning area. Suitable actions are taken by an agent in order to increase the reward in a given environment. Reinforcement learning is different from supervised learning, in reinforcement learning, there is no trained dataset but the agent learns from its experience and decides what actions should be performed for the given task in order to achieve the maximum reward.1. Agent: An individual that can explore the environment and act accordingly. 2. Environment: A state in which an agent is present or bounded by. 3. Action: Actions are the changes/movements taken by an agent within the environment. 4. State: State is a condition returned by the environment after each action taken by the agent. 5. Reward: A response returned to the agent from the environment to estimate the action of the agent. 6.Policy: Policy is an approach applied by the agent for the next action based on the current state.
II. PROBLEM STATEMENT
To consider different types of roads based on the road environment such as accident severity, road type, road surface, light conditions, and junction details and to classify the road structure quality based on the studied dataset from the system using the Q-learning algorithm from Reinforcement learning.
III. RELATED WORKS
Viswanath, Dhanya et.al (2021, ) in the proposed work have studied the relation between the condition of the road, road accidents, and the occurrence of an accident based on environmental factors was studied. They successfully created an application that would predict road accidents based on vehicle type, age of the driver, age of the vehicle, road structure. The dataset has a date, time, and location of accidents, the nature of the accident, whether it was a head-on collision or caused due to over-speed, skidding, or other causes, the type of the road – straight road or a curved road, how many lanes were there, whether it was a junction of multiple roads, the number of fatalities. They have used data mining techniques-Apriori algorithm and Support Vector Machines. The optimized models can be efficiently used by the government to reduce road accidents and to implement policies for road safety. Mariem ALLOUCH and Fedy OUNI (2020, ), their work aimed to highlight the way and extent through which the explanatory variables such as a human, vehicle, and environment affect the severity of a traffic accident. They chose a multinomial logit model to analyze the severity levels of road accidents at intersections.
They used the dataset from the National Road Safety Observatory in Tunisia from the period 2014-2016. The STATA 13.0 statistical software was used to estimate the multinomial logit model. The model developed was able to determine the variables that have a significant effect on the severity of accidents. The advantage of this model is that it allows calculating the probability that a victim is involved in an accident. Jayesh Patil et.al (2020, ), in their work they have used the k-means algorithm and machine learning to examine accident-prone or hotspot areas and their root causes. Government data was used to acquire weather and road conditions to execute their proposed system. Dataset worked with the help of the following attributes: Latitude, longitude, location name, Speed, Time, Alarm type, Weather, Peak hour timings, Potholes, Accident-prone areas. Once testing was completed, the model was made available to local authorities for use.
Hotspot regions were pinned in red where accidents usually occur and locations with high accident-prone areas were shown. Durgesh Kumar Yadav et.al (2020, ), in their work the dataset contained 620 positive videos containing the exact time of the accident in the last 15 frames, and 1130 negative videos containing no accidents. The dataset was collected from six major cities in Taiwan. The dataset includes: 42.6per motorbike to car collision, 19.7per car to car collision, 15per motorbike to motorbike collision, and the remaining 22.7per are of other types. They used Long Short-Term Memory, Convolutional Neural Network, RNN in their project. A module was proposed that detects, and identifies accidents on the occurrence through means of vibrations and sends a notification to the given number.
IV. SYSTEM DESIGN
The System Design Document is for a base-level system, which will work as a proof of concept for the use of building a system that provides a base level of functionality to show feasibility for large-scale production use. The focus is on the generation of the documents and modification of the documents. The system will use in conjunction with other pre-existing systems and will consist largely of a document interaction faced that abstracts document interactions and handling of the document objects. This Document provides the Design specifications of Road Structure Quality Using Reinforcement Learning.
The working principle of road structure quality classification using Reinforcement learning is as shown in figure 3.1. An agent will be placed in an environment, that is to move from the initial state to the final state, by choosing maximum optimization. Based on the road features input will be provided to the agent at the initial state. The agent moves further to the next path, if the move taken by the agent is correct then the agent will be assigned a positive reward and the agent moves forward in order to reach the goal with maximum optimization. If the move taken by the agent is wrong, then a negative reward will be assigned.The goal of an agent is to travel from Path 1 to Path 5 with maximum optimization. From Path 2 the agent moves to Path 3 to reach the destination, if the move is correct the agent is rewarded with +1. From Path 3 the next move agent can take is Path 1, Path 2, or Path 4 to reach the destination. If the agent takes Path 2 it will be rewarded -1 because this path is already covered by the agent. If the agent takes Path 1 or Path 4 it will be rewarded +1. The agent can take either of the paths which has maximum optimization to reach Final state Path 5.The agent will classify the road based on the quality of the road structure.
A. Design Methodology
Figure 2. shows the methodology design of the system. In Reinforcement learning we use Q-learning. The Road environment dataset is collected for which pre-processing of data is carried out. Once pre-processing is done the dataset will be classified as training and testing data. After train and test of data Environment Windygridworld is used in which we define policy, actions and reward for the system. Analysis of classification is done and we test Q-learning with enhanced-Q learning. Finally we will obtain accuracy and confusion matrix for the proposed system.
C. Defining Policy
Table 2. describes the policy defined during the creation of an environment. We have considered 6 features to understand how policy is defined during creating an environment. The 6 features are Accident severity,Road type, Junction detail, Lightconditions, Weather conditions, Road surface. As we can see the integer values in the policy they are the road environment description.
The algorithm is the process to set rules to heed in the calculations and problem-solving solutions to the given requirement and design, which allows the developer to conduct the sequence of actions that need to be considered and help to develop the system in a logical way of representation. Input is the state and actions given to the agent at the starting point. Output is the classification of road quality from the given road environment dataset. In this system, the policy is defined as shown in table 2. The actions are the values of the features., Finally, the reward is the classification of the road quality by saying how good is the road to be used.
Classification of road structure quality is done using a Reinforcement learning algorithm in which Q-learning is used. The model will learn a dataset based on road structure analyze the data and classifies the road quality. Initially, the collection of a dataset and pre-processing are done after which training and testing are carried out. A wind grid environment is created in which policy, action, and rewards are defined. Q-learning and Enhanced Q-learning are used to analyze the dataset. Accuracy and confusion matrix is calculated, where accuracy for q-learning is 93.62% and accuracy for enhanced q-learning is 93.75% is obtained. Classification of a dataset is done based on the quality of road structure environment.
 Viswanath, D., Preethi, K., Nandini, R. and Bhuvaneshwari, R., “A Road Accident Prediction Model Using Data Mining Techniques”, In proceedings of the 5th International Conference on Computing Methodologies and Communication, PP:1618-1623, 2021.  Allouch, M. and Ouni, F., “Modeling the severity of road accidents at intersection”, In proceedings of the IEEE 13th International Colloquium of Logistics and Supply Chain Management, PP : 1-5, 2020.  Patil, J., Prabhu, M., Walavalkar, D. and Lobo, V. B., “Road Accident Analysis using Machine Learning”, In the proceedings of the IEEE Pune Section International Conference, PP: 108- 112, 2020.  Yadav, D. K. and Anjum, I., “Accident Detection Using Deep Learning”. In proceedings of the 2nd International Conference on Advances in Computing, Communication Control and Networking, PP: 232-23, 2020.  Afework, A. and Sipos, T., “Modelling of accidents for four lane non-urban highways using artificial neural networks technique”. In proceedings of the 14th International symposium on applied computational intelligence and informatics, PP: 000047-000052,2020.  Govada, K. A., Jonnalagadda, H. P., Kapavarapu, P., Alavala, S. and Vani, K. S. “ Road Deformation Detection”. In proceedings of the 7th International Conference on Smart Structures and Systems PP: 1-5, 2020.  Parveen, N., Ali, A. and Ali, A. “IOT Based Automatic Vehicle Accident Alert System”. In proceedings of the 5th International Conference on Computing Communication and Automation, PP: 330-333, 2020.  Yellamma, P., Chandra, N. S. N. S. P., Sukhesh, P., Shrunith, P. and Teja, S. S. “Arduino Based Vehicle Accident Alert System Using GPS, GSM and MEMS Accelerometer”. In proceedings of the 5th International Conference on Computing Methodologies and Communication, PP: 486-491, 2020.  Vijay, D. and Chikyal, N. “Vehicle Accident Alert System Using LABVIEW”. In proceedings of the International Conference on Emerging Smart Computing and Informatics PP: 24-27, 2021  Rajesh, G., Benny, A. R., Harikrishnan, A., Abraham, J. J. and John, N. P. “A Deep Learning based Accident Detection System”. In proceedings of International Conference on Communication and Signal Processing, PP: 1322-1325, 2020.  Takanashi, M., Ishii, Y., Sato, S. I., Sano, N., and Sanda, K. “Road-Deterioration Detection using Road Vibration Data with Machine-Learning Approach”. In proceedings of the IEEE International Conference on Prognostics and Health Management, PP: 1-7, 2020.  Shivaanivarsha, N., Niththish, A., and Jagadeesh, U. “A Novel Approach for Monitoring Road Way Surface Using CNN Classifier”. In proceedings of the International Conference on Power, Energy, Control and Transmission Systems, PP: 1-5, 2020.  Chitale, P., Dhope, T., Akhelikar, A. and Dholay, S. “Smart Accident Recognition and Alerting System for Edge Devices”. In proceedings of the 5th International Conference on Computing Communication and Automation, PP: 486-491, 2020.  Kumar, N., Barthwal, A., Lohani, D. and Acharya, D. “Modeling IoT enabled automotive system for accident detection and classification”. In proceedings of the Sensors Applications Symposium, PP: 1-6, 2020.  Li, F., and Jiang, K. “Application of Random-parameter Negative Binomial Model to Examine the Relationship between the Severity of Traffic Accident”. In proceedings of the 5th International Conference on Intelligent Transportation Engineering PP: 351-354, 2020.  Kumar, M. N., Kumar, S. P., Premkumar, R., & Navaneethakrishnan, L. “Smart Characterization of Vehicle Impact and Accident Reporting System”. In proceedings of the 7th International Conference on Advanced Computing and Communication Systems, Vol: 1, PP: 964-968, 2021.  Sewwandi, A. K. T., Dissanayake, D. M. K. P., Navanjani, D. H. K. H., Shangavie, R., Rankothge, W. H., and Gamage, N. “SmartCop: An Automated Platform to Mitigate the Impact of Road Accidents”. In proceedings of the 8th R10 Humanitarian Technology Conference, PP: 1-6, 2020.  Rahman, W., Ruman, M. R., Jahan, K. R., Roni, M. J., Rahman, M. F. and Shahriar, M. A. H. “Vehicle Speed Control and Accident Avoidance System Based on Arm M4 Microprocessor”. In proceedings of the International Conference on Industry 4.0 Technology, PP: 154-158, 2020.  Kumar, N., Acharya, D. and Lohani, D, “An IoT-based vehicle accident detection and classification system using sensor fusion”, IEEE Internet of Things Journal, vol: 8, issue:2, PP:869-880, 2020  Rana, S., Sengupta, S., Jana, S., Dan, R., Sultana, M. and Sengupta, D, “ Prototype Proposal for Quick Accident Detection and Response System”. In proceedings of the Fifth International Conference on Research in Computational Intelligence and Communication Networks, PP: 191-195, 2020.  Kapilan, K., Bandara, S. and Dammalage, T. “Vehicle Accident Detection and Warning System for Sri Lanka Using GNSS Technology”. In proceedings of the International Conference on Image Processing and Robotics, PP: 1-5, 2020.  Song, Y., Wu, P., Gilmore, D. and Li, Q. “A spatial heterogeneity-based segmentation model for analyzing road deterioration network data in multi-scale infrastructure systems”. IEEE Transactions on Intelligent Transportation Systems, 2020.  Hasib, K. M., Showrov, M. I. H. and Das, A. “Accidental prone area detection in bangladesh using machine learning model”. In proceedings of the 3rd International Conference on Computer and Informatics Engineering, PP: 58-62, 2020.  Patil, J., Patil, V., Walavalkar, D. and Lobo, V. B. “Road Accident Analysis and Hotspot Prediction using Clustering”. In proceedings of the 6th International Conference on Communication and Electronics Systems, PP: 763-768, 2021.
Copyright © 2022 Shilpasree S. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Authors : Shilpasree S
Paper Id : IJRASET47751
Publish Date : 2022-11-29
ISSN : 2321-9653
Publisher Name : IJRASET
DOI Link : Click Here