The prospects for vehicle mobility will focus on complete automation. Connected systems is the technology which is booming, making a common man’s life simple, be it in home environment, industries, public spaces or in transportation. The technology which is currently gaining pace in research is on Vehicle-to-Vehicle (V2V) Technology. This involves a network of motor vehicles that continuously access information from or sharing information to the peer vehicles which are driving along. This data exchange can help drivers be alerted to such as possibility of accidents, obstacle detection and presence of traffic gridlocks during travel. Researchers are facing certain hurdles while developing optimum algorithms related to sub-domains of V2V. Amidst these challenges, one is on adjusting the channel capacity. When data is sent, it must be transmitted in a way that no data gets corrupted or lost. Since the environment of vehicle movement is rapidly dynamic, there involves a lot of decision making at every small instances of time. Hence, with the help of previously observed scenarios, performance parameters and Reinforcement Learning (RL) concepts, the decision making could be manageable. The project focuses on managing data rate and transmission power value for every message sent for various scenarios that aids in channel being adequately used. The vehicle traffic is simulated running on an intersection scenario with DSRC Communication. Certain performance metrics such as Packet Delivery Ratio, Channel Busy Ratio is considered to verify the efficiency of the RL model and compared with existing one.
Introduction
Introduction to V2V Communication
V2V communication is a wireless technology enabling vehicles to exchange information (e.g., speed, location) within a 300-meter range.
It helps detect hazards, reduce accidents, and improve traffic flow.
Benefits extend to business logistics, enhancing route efficiency, customer satisfaction, and operational safety.
2. Technology & Standards
Uses DSRC (Dedicated Short Range Communications) and standards from IEEE and SAE (e.g., SAE J2945/1, IEEE 802.11p).
Communication involves Basic Safety Messages (BSMs) containing data such as speed, location, acceleration, and brake status.
CSMA/CA protocol handles wireless channel access to avoid collisions, though not perfectly.
3. Challenges in High Traffic
Congestion and packet loss become issues in dense traffic areas, impacting situational awareness and safety.
Congestion stems from limited channel bandwidth and all vehicles broadcasting BSMs at high frequency and power.
4. Literature Review & Solutions
a. Coordination and Control for CAVs
A bilevel decentralized system optimizes CAV behavior at intersections for energy efficiency and safety.
b. Machine Learning for Congestion Management
DNN models optimize data rate and power to reduce congestion while maintaining performance.
Stochastic geometry used to model and mitigate intersection interference through broadcast-rate optimization.
c. Resource Allocation Techniques
Factor graph models and Belief Propagation help allocate Resource Blocks (RBs) and optimize power, increasing throughput.
d. Channel Modeling
Accurate path loss models and multipath component analysis at intersections reveal signal degradation due to obstructions.
e. Advanced DRL Solutions
Reinforcement Learning (RL) models like DDPG and Ra-DPC manage power and data rate dynamically for better packet delivery and interference control.
f. Optimization in 5G Networks
Joint mode selection and power control using Deep RL (e.g., DDQN) maximize V2V/V2I capacity in decentralized setups.
g. Multi-Objective Optimization
A new MOO framework balances trade-offs in throughput, delay, and QoS for multiple V2V links under environmental uncertainties.
h. Transmit Power Optimization
A DNN-based approach optimizes transmit power for V2V/CUE links, outperforming NP-hard traditional methods in both efficiency and accuracy.
5. Methodology
Reinforcement Learning (RL) is used to adapt data rate and power in real-time at intersections, where congestion is critical.
Key parameters considered include:
Vehicle speed
Orientation
Nearby vehicle density
Equations are used to calculate:
Interference distribution
Probability of successful transmission
Mean number of successful receivers
Power control employs Lagrange Dual Decomposition to avoid power overuse and optimize throughput.
6. DSRC Standards Overview
V2V communication relies on:
BSMs (per SAE J2735)
Positioning subsystems (e.g., GNSS)
DSRC radios using IEEE 1609 WAVE protocols and 802.11p
Enables 360-degree awareness and early warnings even under low visibility or NLOS (non-line-of-sight) scenarios.
DSRC spectrum: 5.85–5.925 GHz, with channels designated for control, safety, and public safety by the FCC.
7. Final Insights
Existing research focuses heavily on highway or straight-lane traffic. This work uniquely emphasizes intersection behavior.
The RL model proposed adapts data rate and power dynamically to optimize safety and channel usage at intersections.
Conclusion
The project demonstrates the benefit of solving the NP-hard conventional problem of finding the optimal data rate and power values for the BSM to be transmitted by the ego/target vehicle to other surrounding vehicles. Here, using MATLAB’s RL Designer toolbox and LTEV2VSim to act as a vehicle simulator, the decision making of RL model is observed. The proposed RL model was compared with the existing work of using BBPSO algorithm as an optimization method to improve the vehicle’s decision taking on the message transmission parameters. The results found out that the proposed model had performed better in PDR and CBR was efficiently used within the suitable limits.The work could be further taken up to check the RL model’s feasibility in working efficiently in various random intersection scenarios and traffic models.
References
[1] S. A. Ahmad, A. Hajisami, H. Krishnan, F. Ahmed-Zaid and E. Moradi-Par, \"V2V System Congestion Control Validation and Performance,\" in IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 2102-2110, March 2019, doi: 10.1109/TVT.2019.2893042.
[2] B. Chalaki and A. A. Malikopoulos, \"Optimal Control of Connected and Automated Vehicles at Multiple Adjacent Intersections,\" in IEEE Transactions on Control Systems Technology, vol. 30, no. 3, pp. 972-984, May 2022, doi: 10.1109/TCST.2021.3082306.
[3] J. Aznar-Poveda, A. -J. Garcia-Sanchez, E. Egea-Lopez and J. García-Haro, \"Simultaneous Data Rate and Transmission Power Adaptation in V2V Communications: A Deep Reinforcement Learning Approach,\" in IEEE Access, vol. 9, pp. 122067-122081, 2021, doi: 10.1109/ACCESS.2021.3109422.
[4] T. Kimura and H. Saito, \"Theoretical Broadcast Rate Optimization for V2V Communications at Intersection,\" in IEEE Transactions on Mobile Computing, vol. 21, no. 9, pp. 3360-3372, 1 Sept. 2022, doi: 10.1109/TMC.2021.3051956.
[5] Y. Hou, X. Wu, X. Tang, X. Qin and M. Zhou, \"Radio Resource Allocation and Power Control Scheme in V2V Communications Network,\" in IEEE Access, vol. 9, pp. 34529-34540, 2021, doi: 10.1109/ACCESS.2021.3061711.
[6] M. Yang et al., \"Dynamic V2V Channel Measurement and Modeling at Street Intersection Scenarios,\" in IEEE Transactions on Antennas and Propagation, vol. 71, no. 5, pp. 4417-4432, May 2023, doi: 10.1109/TAP.2023.3249101.
[7] Yoon, Y., Lee, H. and Kim, H. (2023), “Deep reinforcement learning-based dual-mode congestion control for cellular V2X environments”, Electronics Letters, vol. 59, no. 20, pp. e12984. doi: https://doi.org/10.1049/ell2.12984
[8] Wooyeol Yang, Han-Shin Jo, “Deep-reinforcement-learning-based range-adaptive distributed power control for cellular-V2X”, ICT Express, Volume 9, Issue 4, 2023, pp. 648-655, ISSN 2405-9595, https://doi.org/10.1016/j.icte.2022.07.008.
[9] D. Zhao, H. Qin, B. Song, Y. Zhang, X. Du and M. Guizani, \"A Reinforcement Learning Method for Joint Mode Selection and Power Adaptation in the V2V Communication Network in 5G,\" in IEEE Transactions on Cognitive Communications and Networking, vol. 6, no. 2, pp. 452-463, June 2020, doi: 10.1109/TCCN.2020.2983170.
[10] M. Ma, C. Wang, Z. Li and G. Min, \"A Multi-Objective Optimization-Based Transmission Design for V2V Communication Networks,\" in IEEE Transactions on Vehicular Technology, vol. 72, no. 10, pp. 13081-13093, Oct. 2023, doi: 10.1109/TVT.2023.3274780.
[11] D. Ron and J. -R. Lee, \"DNN-Based Dynamic Transmit Power Control for V2V Communication Underlaid Cellular Uplink,\" in IEEE Transactions on Vehicular Technology, vol. 71, no. 11, pp. 12413-12418, Nov. 2022, doi: 10.1109/TVT.2022.3192833.
[12] Wang Yanbin, Wu Zhuofei, Zhao Jing, Li Zhijuan, Ma Xiaomin, “Analysis and adaptive optimization of vehicular safety message communications at intersections”, Ad Hoc Networks, Volume 107, 2020, 102241, ISSN 1570-8705, https://doi.org/10.1016/j.adhoc.2020.102241.
[13] “V2V simulators and related software”, RIT Wireless and IoT Security and Privacy Lab, Article, Published on: Dec. 23, 2021 [Accessed on: Jan. 25, 2024]
[14] Abid Ali Awan, “An Introduction to Q-learning: A Tutorial For Beginners,” Datacamp, Article, Published on Oct. 2022, Link: https://www.datacamp.com/tutorial/introduction-q-learning-beginner-tutorial
[15] “Train Reinforcement Learning in MDP Environment”, Mathworks Documentation, Link: https://in.mathworks.com/help/reinforcement-learning/ug/train-reinforcement-learning-agent-in-mdp-environment.html
[16] “Q-learning Agents”, Reinforcement Learning Toolbox, Mathworks Documentation, Link: https://in.mathworks.com/help/reinforcement-learning/ug/q-learning-agents.html
[17] “Reinforcement Leaning Onramp”, MATLAB Online Training, Online Self-paced course, Link: https://matlabacademy.mathworks.com/details/reinforcement-learning-onramp/reinforcementlearning
[18] LTEV2Vsim, GitHub, Alessandro Bazzi, https://github.com/alessandrobazzi/LTEV2Vsim [Website]
[19] “SAE Standard Overview of Basic Safety Message (BSM)”, Farid Ahmed-Zaid, Ford Motor Company, August 14, 2019, https://www.tomesoftware.com/wp-content/uploads/2019/08/5-2019-B2V-Workshop-Detroit-Farid-Ahmed-Zaid-BSM-messages.pdf
[20] “On-Board System Requirements for V2V Safety Communications J2945/1_201603”, SAE International,
https://www.sae.org/standards/content/j2945/1_201603/
[21] “Dedicated Short Range Communication (DSRC) Systems Engineering Process Guidance for SAE J2945/X Documents and Common Design Concepts™ J2945_201712”, SAE International, https://www.sae.org/standards/content/j2945_201712/
[22] Mike Wang, “Deep Q-learning Tutorial: minDQN, A Practical Guide to Deep Q-Networks”, Article, Towards Data Science, Published on Nov. 18th, 2020, [Last accessed on: April 3rd , 2024], Link: https://towardsdatascience.com/deep-q-learning-tutorial-mindqn-2a4c855abf