Real-Time Lane Detection for Autonomous Driving

Abstract

Lane detection stands as a foundational capability for modern autonomous vehicles and advanced driver assistance systems (ADAS). Despite decades of progress, the problem is far from solved — road environments remain unpredictable, lane markings vary dramatically across geographies, and real-time constraints impose strict computational budgets on embedded hardware. This paper offers a structured review of the state of the art in deep learning-based lane detection, tracing the field from early convolutional segmentation approaches through the latest transformer-based architectures. We examine the progression from pixel-level classification frameworks to anchor-driven regression models, polynomial curve fitting strategies, and end-to-end detection transformers. We further survey the benchmark datasets and evaluation metrics that shape how the community measures progress, and critically assess persistent challenges including adverse weather, occlusion, non-standard lane markings, and the sim-to-real gap. Drawing on these findings, we identify directions that are likely to define the next generation of lane detection systems, including multi-task learning, domain-adaptive training, 3D lane reconstruction, and lightweight deployment on embedded automotive platforms.

Introduction

a comprehensive review of lane detection systems for autonomous driving, explaining why the problem is difficult, how solutions have evolved, and what challenges remain.

Lane detection is essential for autonomous vehicles to understand road structure and maintain safe navigation. However, it is a challenging task due to real-world conditions like poor lighting, rain, fog, occlusions from vehicles, faded lane markings, and the need for real-time processing under limited hardware constraints.

Early methods relied on hand-crafted computer vision techniques such as Hough Transform, RANSAC, and Kalman filters, but these struggled in complex environments. The field shifted significantly with deep learning, especially CNN-based approaches that learn features automatically from data.

Modern methods are grouped into several categories:

• Segmentation-based methods (e.g., SCNN) treat lanes as pixel-wise classification problems but struggle with long-range lane continuity.
• Anchor-based/regression methods (e.g., LaneATT, CLRNet) directly predict lane shapes and improve speed and accuracy.
• Curve-based methods (polynomials, Bezier, B-splines) represent lanes as mathematical curves for compact modeling.
• Keypoint-based methods detect lane points and connect them using graph-based reasoning.
• Transformer-based methods (e.g., LSTR, LDTR, O2SFormer) use attention mechanisms for end-to-end lane prediction without heavy post-processing.

The review also discusses major datasets like TuSimple (simple highway scenes), CULane (complex urban conditions), and LLAMAS (high-quality annotations), along with newer robustness benchmarks like LanEvil.

Performance is measured using metrics such as precision, recall, F1-score, and FPS, balancing accuracy with real-time efficiency.

Finally, the text highlights ongoing challenges, especially adverse weather conditions, lighting issues, and real-world robustness gaps, noting that future solutions may require multi-modal sensing (camera + LiDAR/radar) and more robust deep learning architectures.

Conclusion

This review has traced the development of deep learning-based lane detection from its origins in semantic segmentation through four broad paradigm shifts: from pixel classification to anchor-driven regression, from single-point representations to curve fitting and keypoint grouping, and most recently to end-to-end transformer architectures that eliminate post-processing entirely. Along the way, the community has produced a rich collection of benchmark datasets and standardized evaluation metrics that allow meaningful comparison across approaches. The best current methods — CLRNet, LDTR, O2SFormer, and their contemporaries — achieve impressive results on standard benchmarks, but important gaps remain. Adverse weather, complex intersection geometry, the sim-to-real transfer problem, and the tension between accuracy and deployable inference speed all represent open challenges with clear practical significance. The near-term research agenda is likely to focus on several areas simultaneously: 3D lane detection to handle non-flat roads, multi-task frameworks to improve efficiency, temporal modeling to improve stability, and domain adaptation to close the performance gap between the lab and the real world. Foundation models trained at internet scale may play a growing role as transfer learning substrates, reducing dependence on large domain-specific labeled datasets. The ultimate measure of progress in lane detection is not a benchmark score but a vehicle that drives safely, consistently, and in conditions that no one thought to include in the training data. That goal remains ahead of us, but the trajectory of progress gives good reason for optimism.

References

[1] Duda, R. O., & Hart, P. E. (1972). Use of the Hough transformation to detect lines and curves in pictures. Communications of the ACM, 15(1), 11–15. [2] Borkar, A., Hayes, M., & Smith, M. T. (2009). Robust lane detection and tracking with RANSAC and Kalman filter. Proceedings of the IEEE International Conference on Image Processing, 3261–3264. [3] Zou, Q., Jiang, H., Dai, Q., Yue, Y., Chen, L., & Wang, Q. (2019). Robust lane detection from continuous driving scenes using deep neural networks. IEEE Transactions on Vehicular Technology, 69(1), 41–54. [4] Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE/CVF CVPR, 3431–3440. [5] Pan, X., Shi, J., Luo, P., Wang, X., & Tang, X. (2018). Spatial as deep: Spatial CNN for traffic scene understanding. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1). [6] Li, X., Li, J., Hu, X., & Yang, J. (2019). Line-CNN: End-to-end traffic line detection with line proposal unit. IEEE Transactions on Intelligent Transportation Systems, 21(1), 248–258. [7] Tabelini, L., Berriel, R., Paixão, T. M., Badue, C., De Souza, A. F., & Oliveira-Santos, T. (2021). Keep your eyes on the lane: Real-time attention-guided lane detection. Proceedings of the IEEE/CVF CVPR, 294–302. [8] Zheng, T., Huang, Y., Liu, Y., Tang, W., Yang, Z., Cai, D., & He, X. (2022). CLRNet: Cross layer refinement network for lane detection. Proceedings of the IEEE/CVF CVPR, 898–907. [9] Tabelini, L., Berriel, R., Paixão, T. M., Badue, C., De Souza, A. F., & Oliveira-Santos, T. (2020). PolyLaneNet: Lane estimation via deep polynomial regression. Proceedings of the International Conference on Pattern Recognition (ICPR). [10] Feng, Z., Guo, S., Tan, X., Xu, K., Wang, M., & Ma, L. (2022). Rethinking efficient lane detection via curve modeling. Proceedings of the IEEE/CVF CVPR, Vancouver. [11] Liu, L., Chen, X., Zhu, S., & Tan, P. (2021). CondLaneNet: A top-to-down lane detection framework based on conditional convolution. Proceedings of the IEEE/CVF ICCV, 3773–3782. [12] Yang, Z., Shen, C., Shao, W., et al. (2024). LDTR: Transformer-based lane detection with anchor-chain representation. Computational Visual Media, 10, 753–769. [13] Zhou, K., & Zhou, R. (2023). End-to-end lane detection with one-to-several transformer (O2SFormer). arXiv preprint arXiv:2305.00675. [14] Zhang, T., Wang, L., Li, H., et al. (2024). LanEvil: Benchmarking the robustness of lane detection to environmental illusions. Proceedings of the 32nd ACM International Conference on Multimedia, 5403–5412. [15] Luo, X., Huang, Y., Cui, J., et al. (2025). Deep learning-based lane detection for intelligent driving: A comprehensive survey of methods, datasets, challenges and outlooks. Neurocomputing, ScienceDirect. [16] Wang, Z., Ren, W., & Qiu, Q. (2018). LaneNet: Real-time lane detection networks for autonomous driving. arXiv preprint arXiv:1807.01726. [17] Wang, D., Wen, C., Pan, Y., & Shen, C. (2022). End-to-end monocular 3D lane detection via differentiable geometry distillation. Proceedings of the IEEE/CVF ECCV. [18] He, X., Guo, H., Zhu, K., et al. (2024). Monocular lane detection based on deep learning: A survey. arXiv preprint arXiv:2411.16316. [19] Cai, Z., et al. (2024). Lane2Seq: Towards unified lane detection via sequence generation. Proceedings of the IEEE/CVF CVPR, 2024. [20] Maddiralla, V., & Subramanian, S. (2024). Effective lane detection on complex roads with convolutional attention mechanism in autonomous vehicles. Scientific Reports, 14, 19193.

Copyright

Copyright © 2026 Ritu Kumar Jha, Vishal Kumar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.