Home >Projects >Survey Deep Learning

A Survey of Deep Learning Techniques for Autonomous Driving

Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias and Gigel Macesanu
24 June 2019

Img   Img   Img   Img


Img   Img   Img   Img

Deep Learning Based Decision-making Architectures used in Self-driving Cars

A survey of motion planning and control techniques for self-driving urban vehicles [pdf] IEEE Trans. Intelligent Vehicles, 2016
Brian Paden, Michal Cap, Sze Zheng Yong, Dmitry Yershov, Emilio Frazzoli

Overview of Deep Learning Technologies

Deep Convolutional Neural Networks
Gradient-based learning applied to document recognition [pdf] Proceedings of the IEEE, 1998
Yann LeCun Leon Bottou Yoshua Bengio and Patrick Haffnerner
Representation learning: A review and new perspectives [pdf] IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
Yoshua Bengio, Aaron Courville, Pascal Vincent
Rapid object detection using a boosted cascade of simple features [pdf] IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001
Paul Viola, Michael Jones
A comparative study of texture measures with classification based on featured distributions [pdf] Pattern Recognition, 1996
Timo Ojala, Matti Pietiäinen, David Harwood
Histograms of oriented gradients for human detection [pdf] In CVPR, 2005
Navneet Dalal, Bill Triggs
Shape and arrangement of columns in cats striate cortex [pdf] The Journal of Physiology, 1963
David H. Hubel, Torsten N. Wiesel
Separate visual pathways for perception and action [pdf] Trends in Neurosciences, 1992
Melvyn A. Goodale, A. David Milner
Imagenet classification with deep convolutional neural networks [pdf] Advances in Neural Information Processing Systems 25, 2012
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
Parallel Distributed Processing: Explorations in the Microstructure of Cognition [pdf] MIT Press, 1986
D. E. Rumelhart, J. L. McClelland, C. PDP Research Group
Recurrent Neural Networks
Long short-term memory [pdf] Neural computation, 1997
Sepp Hochreiter, Jürgen Schmidhuber
Deep Reinforcement Learning
Introduction to Reinforcement Learning [pdf] MIT Press, 1998
Richard Sutton, Andrew G. Barto
Dynamic Programming [pdf] Princeton University Press, 1957
Richard Bellman
Q-learning [pdf] Machine Learning, 1992
Christopher Watkins, Peter Dayan
Human-level control through deep reinforcement learning [pdf] Nature, 2015
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, D. Hassabis
Rainbow: Combining improvements in deep reinforcement learning [pdf] 2017
Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver
Deep reinforcement learning framework for autonomous driving [pdf] Electronic Imaging, 2017
Ahmad El Sallab, Mohammed Abdou, Etienne Perot, Senthil Yogamani
Continuous Control with Deep Reinforcement Learning [pdf] Int. Conf. on Learning Representations ICLR, 2016
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, Daan Wierstra
Continuous Deep Q-Learning with Model-based Acceleration [pdf] Int. Conf. on Machine Learning ICML, 2016
Gu Shixiang, Lillicrap Timothy, Sutskever Ilya, Levine Sergey
End-to-End Race Driving with Deep Reinforcement Learning [pdf] Int. Conf. on Robotics and Automation ICRA 2018
Jaritz Maximilian, Charette Raoul, Toromanoff Marin, Perot Etienne, Nashashibi Fawzi
Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments [pdf] CoRR, 2016
Markus Wulfmeier, Dominic Zeng Wang, Ingmar Posner

Deep Learning for Driving Scene Perception and Localization

Overview of environment perception for intelligent vehicles [pdf] IEEE Transactions on Intelligent Transportation Systems, 2017
Hao Zhu, Ka-Veng Yuen, Lyudmila Mihaylova, Henry Leung
Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art [pdf] CoRR, 2017
Janai, Joel and Guney, Fatma and Behl, Aseem and Geiger, Andreas
Camera Based Environment Perception
ImageNet Large Scale Visual Recognition Challenge [pdf] International Journal of Computer Vision (IJCV), 2015
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei
Object detection with deep learning: A review [pdf] arXiv preprint arXiv, 2018
Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu
Recent advances in object detection in the age of deep convolutional neural networks [pdf] arXiv preprint arXiv, 2018
Shivang Agarwal, Jean Ogier du Terrail, Frederic Jurie
You only look once: Unified, real-time object detection [pdf] Proceedings of the IEEE conference on computer vision and pattern recognition, 2016
Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi
Ssd: Single shot multibox detector [pdf] European conference on computer vision, 2016
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg
Yolo9000: better, faster, stronger [pdf] arXiv preprint, 2017
Joseph Redmon, Ali Farhadi
YOLOv3: An Incremental Improvement [pdf] arXiv preprint, 2018
Joseph Redmon, Ali Farhadi
CornerNet: Detecting Objects as Paired Keypoints [pdf] Proceedings of the European Conference on Computer Vision (ECCV), 2018
Hei Law, Jia Deng
Single-shot refinement neural network for object detection [pdf] arXiv preprint, 2017
Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, Stan Z. Li
Rich feature hierarchies for accurate object detection and semantic segmentation [pdf] Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014
Ross Girshick, Jeff Donahue, Trevor Darrell, Jitendra Malik
Faster r-cnn: towards real-time object detection with region proposal networks [pdf] IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017
Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun
Fast R-CNN [pdf] Proceedings of the IEEE international conference on computer vision, 2015
Ross Girshick
R-fcn: Object detection via regionbased fully convolutional networks [pdf] Advances in neural information processing systems, 2016
Jifeng Dai, Yi Li, Kaiming He, Jian Sun
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size [pdf] arXiv preprint, 2016
Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, Kurt Keutzer
An efficient object detection algorithm based on compressed networks [pdf] Symmetry, 2018
Jianjun Li, Kangjian Peng, Chin-Chen Chang
ImageNet Classification with Deep Convolutional Neural Networks [pdf] Advances in Neural Information Processing Systems, 2012
Alex Krizhevsky, Sutskever Ilya, Hinton Geoffrey E
A Review on Deep Learning Techniques Applied to Semantic Segmentation [pdf] CoRR, 2017
Alberto Garcia-Garcia, Sergio Orts-Escolano, Sergiu Oprea, Victor Villena-Martinez, Jose Garcia-Rodriguez
Imagenet classification with deep convolutional neural networks [pdf] Advances in Neural Information Processing Systems, 2012
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton
Very deep convolutional networks for large-scale image recognition [pdf] arXiv preprint, 2014
Karen Simonyan, Andrew Zisserman
Going Deeper with Convolutions [pdf] 2014
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich
Deep Residual Learning for Image Recognition [pdf] Proceedings of the IEEE conference on computer vision and pattern recognition, 2016
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
Segnet: A deep convolutional encoder-decoder architecture for image segmentation [pdf] CoRR, 2015
Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla
Icnet for real-time semantic segmentation on high-resolution images [pdf] arXiv preprint, 2017
Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, Jiaya Jia
Speeding up semantic segmentation for autonomous driving [pdf] 2016
Michael Treml, José Arjona-Medina, Thomas Unterthiner, Rupesh Durgesh, Felix Friedmann, Peter Schuberth, Andreas Mayr, Martin Heusel, Markus Hofmarcher, Michael Widrich, Bernhard Nessler, Sepp Hochreiter
Learning to refine object segments [pdf] European Conference on Computer Vision, 2016
Pedro O. Pinheiro, Tsung-Yi Lin, Ronan Collobert, Piotr Dollàr
Enet: A deep neural network architecture for real-time semantic segmentation [pdf] arXiv preprint, 2016
Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello
Adapnet: Adaptive semantic segmentation in adverse environmental conditions [pdf] IEEE International Conference on Robotics and Automation (ICRA), 2017
Abhinav Valada, Johan Vertens, Ankit Dhall, Wolfram Burgard
Mask R-CNN [pdf] 2017 IEEE International Conference on Computer Vision (ICCV)
Kaiming He, Georgia Gkioxari, Piotr Dollar, Ross B. Girshick
Simultaneous Localization and Mapping: A Survey of Current Trends in Autonomous Driving [pdf] IEEE Transactions on Intelligent Vehicles 2017
G. Bresson, Z. Alsayed, L. Yu, S. Glaser
Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments [pdf] 2018 IEEE International Conference on Robotics and Automation (ICRA)
Dan Barnes, Will Maddern, Geoffrey Pascoe, Ingmar Posner
PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization [pdf] Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV)
Kendall Alex, Grimes Matthew, Cipolla Roberto
Image-Based Localization Using LSTMs for Structured Feature Correlation [pdf] 2017 IEEE International Conference on Computer Vision (ICCV)
Florian Walch, Caner Hazirbas, Laura Leal-Taix, Torsten Sattler, Sebastian Hilsenbeck, Daniel Cremers
Image-Based Localization Using Hourglass Networks [pdf] 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)
Iaroslav Melekhov, Juha Ylioinas, Juho Kannala, Esa Rahtu
Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network [pdf] The IEEE International Conference on Computer Vision (ICCV)
Laskar Zakaria, Melekhov Iaroslav, Kalia Surya, Kannala Juho
Learning Less is More – 6D Camera Localization via 3D Surface Regression [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2018
Eric Brachmann, Carsten Rother
VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry [pdf] IEEE Robotics and Automation Letters 2018
Noha Radwan, Abhinav Valada, Wolfram Burgard
Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization [pdf] Proc. of the 2nd Conference on Robot Learning (CoRL)
Paul-Edouard Sarlin, Frédéric Debraine, Marcin Dymczyk, Roland Siegwart, Cesar Cadena
Camera Based Environment Perception
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2017
Charles Ruizhongtai Qi, Hao Su, Kaichun Mo, Leonidas J. Guibas
VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection [pdf] IEEE Conf. on Computer Vision and Pattern Recognition 2018
Yin Zhou, Oncel Tuzel
Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting With a Single Convolutional Net [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2018
Luo Wenjie, Yang Bin, Urtasun Raquel
Frustum PointNets for 3D Object Detection from {RGB-D} Data [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2018
Charles Ruizhongtai Qi, Wei Liu, Chenxia Wu, Hao Su, Leonidas J. Guibas
Multi-View 3D Object Detection Network for Autonomous Driving [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2017
Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, Tian Xia
RoarNet: A Robust 3D Object Detection based on RegiOn Approximation Refinement [pdf] CoRR 2018
Kiwoo Shin, Youngwook Paul Kwon, Masayoshi Tomizuka
Joint 3d proposal generation and object detection from view aggregation [pdf] International Conference on Intelligent Robots and Systems (IROS) 2018
Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, Steven Lake Waslander
Feature Learning for Scene Flow Estimation from LIDAR [pdf] International Conference on Intelligent Robots and Systems (IROS) 2018
Ushani Arash K., Eustice Ryan M.
Learning to Localize Using a LiDAR Intensity Map [pdf] Proc. of the 2nd Conference on Robot Learning (CoRL), 2018
Ioan Andrei Barsan, Shenlong Wang, Andrei Pokrovsky, Raquel Urtasun
Learning to See the Wood for the Trees: Deep Laser Localization in Urban and Natural Environments on a CPU [pdf] IEEE Robotics and Automation Letters 2019
G. Tinchev, A. Penate-Sanchez, M. Fallon
Camera vs. LiDAR Debate
How Tesla and Waymo are Tackling a Major Problem for Self-Driving Cars: Data [page] online, 2018
Sean O'Kane
Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems [page] online, 2018
SAE Committee
Test Methodology for Rain Influence on Automotive Surround Sensors [pdf] 2016 IEEE 19th Int. Conf. on Intelligent Transportation Systems (ITSC)
Sinan Hasirlioglu, Alexander Kamann, Igor Doric, Thomas Brandmeier
Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving [pdf] IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) 2019
Wang Yan, Chao Wei-Lun, Garg Divyansh, Hariharan Bharath, Campbell Mark, Weinberger Kilian
Perception using occupancy maps
Laser scanner based slam in real road and traffic environment [pdf] IEEE International Conference Robotics and Automation (ICRA09). Workshop on Safe navigation in open and dynamic environments Application to autonomous vehicles, 2009
Olivier Garcia-Favrot, Michel Parent
Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) [pdf] Cambridge: The MIT Press, 2005
S. Thrun, W. Burgard, D. Fox
Deep Grid Net (DGN): A Deep Learning System for Real-Time Driving Context Understanding [pdf] Int. Conf. on Robotic Computing IRC 2019, 2019
Liviu A. Marina, Bogdan Trasnea, Cocias Tiberiu, Andrei Vasilcoi, Florin Moldoveanu, Sorin M. Grigorescu
End-to-end tracking and semantic segmentation using recurrent neural networks [pdf] CoRR, 2016
Peter Ondruska, Julie Dequaire, Dominic Zeng Wang, Ingmar Posner
Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling [pdf] CoRR, 2017
Stefan Hoermann, Martin Bach, Klaus Dietmayer
Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling [pdf] CoRR, 2016
Sebastian Ramos, Stefan Gehrig, Peter Pinggera, Uwe Franke, Carsten Rother
Towards Road Type Classification with Occupancy Grids [pdf] IEEE Intelligent Vehicles Symposium. Workshop: DeepDriving - Learning Representations for IntelligentVehicles, 2016
Christoph Seeger, Andre Muller, Loren Schwarz, and Michael Manz

Deep Learning for Path Planning and Behavior Arbitration

Perception, Planning, Control, and Coordination for Autonomous Vehicles [pdf] Machines, 2017
Scott Drew Pendleton, Hans Andersen, Xinxin Du, Xiaotong Shen, Malika Meghjani, You Hong Eng, Daniela Rus, Marcelo H. Ang Jr.
Planning and Decision-Making for Autonomous Vehicles [pdf] Annual Review of Control, Robotics, and Autonomous Systems, 2018
Wilko Schwarting, Javier Alonso-Mora, Daniela Rus
Human-like Planning of Swerve Maneuvers for Autonomous Vehicles [pdf] IEEE Intelligent Vehicles Symposium (IV), 2016
Tianyu Gu, John M. Dolan, Jin-Woo Lee
NeuroTrajectory: A Neuroevolutionary Approach to Local State Trajectory Learning for Autonomous Vehicles [pdf] IEEE Robotics and Automation Letters, 2019
Sorin Grigorescu, Bogdan Trasnea, Liviu Marina, Andrei Vasilcoi, Tiberiu Cocias
Driving Like a Human: Imitation Learning for Path Planning using Convolutional Neural Networks [pdf] International Conference on Robotics and Automation Workshops, 2017
Eike Rehder, Jannik Quehl, Christoph Stiller
A fast integrated planning and control framework for autonomous driving via imitation learning [pdf] CoRR, 2017
Liting Sun, Cheng Peng, Wei Zhan, Masayoshi Tomizuka
Intelligent Land-Vehicle Model Transfer Trajectory Planning Method Based on Deep Reinforcement Learning [pdf] Sensors (Basel, Switzerland), 2018
Lingli Yu, Xuanya Shao, Yadong Wei, Kaijun Zhou
Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments [pdf] CoRR, 2017
Chris Paxton, Vasumathi Raman, Gregory D. Hager, Marin Kobilarov
Grid path planning with deep reinforcement learning: Preliminary results [pdf] Procedia Computer Science, 2018
A. I. Panov, K. S. Yakovlev, R. Suvorov
Deep Learning Driven Visual Path Prediction from a Single Image [pdf] CoRR, 2016
Siyu Huang, Xi Li, Zhongfei Zhang, Zhouzhou He, Fei Wu, Wei Liu, Jinhui Tang, Yueting Zhuang
Vision-Based Robot Path Planning with Deep Learning [pdf] Computer Vision Systems, 2017
Ping Wu, Yang Cao, Yu Qing He, Decai Li

Motion Controllers for AI-based Self-Driving Cars

Deep Reinforcement Learning Control
Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving [pdf] 2016
Shai Shalev-Shwartz, Shaked Shammah, Amnon Shashua
Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search [pdf] IEEE International Conference on Robotics and Automation (ICRA), 2016
Tianhao Zhang, Gregory Kahn, Sergey Levine, Pieter Abbeel
Learning to drive in a day [pdf] 2018
Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh-Dieu Lam, Alex Bewley, Amar Shah
Learning Controllers
Comparison of lateral controllers for autonomous vehicle: Experimental results [pdf] 2016
Salvador Dominguez, Alan Ali, Gaetan Garcia, Philippe Martinet
Autonomous Automobile Trajectory Tracking for Off-Road Driving: Controller Design, Experimental Validation and Racing [pdf] American Control Conference, 2007
Gabriel M. Hoffmann, Claire J. Tomlin, Michael Montemerlo, Sebastian Thrun
Robust Constrained Learning-based NMPC enabling reliable mobile robot path tracking [pdf] International Journal of Robotics Research, 2016
Chris J. Ostafew, Angela P. Schoellig, Timothy D. Barfoot
Local Gaussian Process Regression for Real Time Online Model Learning and Control [pdf] Proceedings of the neural information processing systems conference, 2008
Duy Nguyen-Tuong, Jan Peters, Matthias Seeger
Efficient Bayesian Local Model Learning for Control [pdf] IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016
Franziska Meier, Philipp Hennig, Stefan Schaal
Learning-Based Nonlinear Model Predictive Control to Improve Vision-Based Mobile Robot Path-Tracking in Challenging Outdoor Environments [pdf] ournal of Field Robotics, 2015
Chris J. Ostafew, Angela P. Schoellig, Timothy D. Barfoot
On-line regression algorithms for learning mechanical models of robots: A survey [pdf] Robotics and Autonomous Systems, 2011
Olivier Sigaud, Camille Salaün, Vincent Padois
Visual Teach and Repeat, Repeat, Repeat: Iterative Learning Control to Improve Mobile Robot Path Tracking in Challenging Outdoor Environments [pdf] 2013
Chris J. Ostafew, Angela P. Schoellig, and Timothy D. Barfoot
Application of iterative learning control in tracking a Dubin’s path in parallel parking [pdf] International Journal of Automotive Technology, 2017
Benjamas Panomruttanarug
Path tracking of highly dynamic autonomous vehicle trajectories via iterative learning control [pdf] 2015 American Control Conference (ACC), 2015
N. R. Kapania, J. C. Gerdes
A novel iterative learning path-tracking control for nonholonomic mobile robots against initial shifts [pdf] International Journal of Advanced Robotic Systems, 2017
Zhao Yang, Fengyu Zhou, Yan Li, Yugang Wang
A Review on Motion Control of Unmanned Ground and Aerial Vehicles Based on Model Predictive Control Techniques [pdf] Engineering Science and Military Technologies, 2018
Mohamed A. Kamel, Ahmed Taimour Hafez, Xiang Yu
A Learning-Based Framework for Velocity Control in Autonomous Driving [pdf] IEEE Transactions on Automation Science and Engineering, 2016
Stéphanie Lefèvre, Ashwin Carvalho, Francesco Borrelli
Autonomous Car Following: A Learning-Based Approach [pdf] IEEE Intelligent Vehicles Symposium (IV), 2015
Stéphanie Lefèvre, Ashwin Carvalho, Francesco Borrelli
Aggressive Deep Driving: Combining Convolutional Neural Networks and Model Predictive Control [pdf] 2017
Paul Drews, Grady Williams, Brian Goldfain, Evangelos A. Theodorou, James M. Rehg
Aggressive deep driving: Model predictive control with a CNN cost model [pdf] 2017
Paul Drews, Grady Williams, Brian Goldfain, Evangelos A. Theodorou, James M. Rehg
Autonomous Racing using Learning Model Predictive Control [pdf] American Control Conference(ACC), 2017
Ugo Rosolia, Ashwin Carvalho, Francesco Borrelli
Repetitive learning model predictive control: An autonomous racing example [pdf] IEEE 56th Annual Conference on Decision and Control (CDC), 2017
Ugo Rosolia, Maximilian Brunner, Francesco Borrelli, Jon Gonzales
Learning-based nonlinear model predictive control to improve vision-based mobile robot path-tracking in challenging outdoor environments [pdf] IEEE International Conference on Robotics and Automation (ICRA), 2014
Chris J. Ostafew, Angela P. Schoellig, Timothy D. Barfoot
Learning Deep Neural Network Control Policies for Agile Off-Road Autonomous Driving [pdf] 2017
Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots
Agile Off-Road Autonomous Driving Using End-to-End Deep Imitation Learning [pdf] 2017
Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots
Agile Autonomous Driving using End-to-End Deep Imitation Learning [pdf] CoRR, 2018
Yunpeng Pan, Ching-An Cheng, Kamil Saigol, Keuntaek Lee, Xinyan Yan, Evangelos Theodorou, Byron Boots
End2End Learning Control
Alvinn: An autonomous land vehicle in a neural network [pdf] Advances in neural information processing systems, 1989
D. A. Pomerleau
Off-Road Obstacle Avoidance through End-to-End Learning [pdf] Advances in neural information processing systems, 2006
Yann LeCun, Urs Muller, Jan Ben, Eric Cosatto, Beat Flepp
Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car [pdf] arXiv preprint, 2017
Mariusz Bojarski, Philip Yeres, Anna Choromanska, Krzysztof Choromanski, Bernhard Firner, Lawrence Jackel, Urs Muller
End-to-end learning of driving models from large-scale video datasets [pdf] arXiv preprint, 2017
Xu Huazhe, Gao Yang, Yu Fisher, Darrell Trevor
End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies [pdf] arXiv preprint, 2017
Hesham M. Eraqi, Mohamed N. Moustafa, Jens Honer
End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners [pdf] European Conference on Computer Vision (ECCV), 2018
Simon Hecker, Dengxin Dai, Luc Van Gool
Learning a deep neural net policy for end-to-end control of autonomous vehicles [pdf] American Control Conference (ACC), 2017
Viktor Rausch, Andreas Hansen, Eugen Solowjow, Chang Liu, Edwin J. Kreuzer, J. Karl Hedrick
DeepPicar: A Low-cost Deep Neural Network-based Autonomous Car [pdf] The 24th IEEE Inter. Conf. on Embedded and Real-Time Computing Systems and Applications (RTCSA), 2018
Michael Garrett Bechtel, Elise McEllhiney, Heechul Yun
Deep Reinforcement Learning framework for Autonomous Driving [pdf] CoRR, 2017
Ahmad El Sallab, Mohammed Abdou, Etienne Perot, Senthil Yogamani
Feature Analysis and Selection for Training an End-to-End Autonomous Vehicle Controller Using the Deep Learning Approach [pdf] CoRR, 2017
Shun Yang, Wenshuo Wang, Chang Liu, Kevin Deng, J. Karl Hedrick
End-to-End Race Driving with Deep Reinforcement Learning [pdf] IEEE International Conference on Robotics and Automation (ICRA), 2018
Maximilian Jaritz, Raoul de Charette, Marin Toromanoff, Etienne Perot, Fawzi Nashashibi
Arguing Machines: Perception-Control System Redundancy and Edge Case Discovery in Real-World Autonomous Driving [pdf] CoRR, 2017
Alex Fridman, Benedikt Jenik, Bryan Reimer
DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving [pdf] IEEE International Conference on Computer Vision (ICCV), 2015
Chenyi Chen, Ari Seff, Alain Kornhauser, Jianxiong Xiao
End-to-End Driving in a Realistic Racing Game with Deep Reinforcement Learning [pdf] IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017
Etienne Perot, Maximilian Jaritz, Marin Toromanoff, Raoul de Charette
Autonomous Vehicle Control via Deep Reinforcement Learning [pdf] Chalmers University of Technology, 2017
Simon Kardell, Mattias Kuosku

Safety of Deep Learning in Autonomous Driving

Engineering safety-critical systems in the 21st century [pdf] 2010
Ferrel. T.
Making the case for safety of machine learning in highly automated driving [pdf] Lecture Notes in Computer Science, 2017
H. C. Burton S., Gauerhof L.
Engineering Safety in Machine Learning [pdf] Information Theory and Applications Workshop (ITA), 2016
Kush R. Varshney
Concrete Problems in AI Safety [pdf] CoRR, 2016
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, Dan Mané
The Concepts of Risk and Safety [pdf] Springer Netherlands, 2012
Niklas Möller
An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software [pdf] SAE Technical Paper, 2017
Rick Salay, Rodrigo Queiroz, Krzysztof Czarnecki
Challenges in applying the iso 26262 for driver assistance systems [pdf] Schwerpunkt Vernetzung, 5. Tagung Fahrerassistenz, 2012
S. Bernd, R. Detlev, E. Susanne, W. Ulf, B. Wolfgang, Patz, Carsten
Humans and automation: Use, misuse, disuse, abuse [pdf] Human Factors, 1997
R. Parasuraman and V. Riley
Safety-Critical Systems [pdf] 2018
José M. Faria
Domain Adaptation for Statistical Classifiers [pdf] J. Artif. Int. Res., 2016
H. Daume III, D. Marcu
Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission [pdf] Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015
R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, and N. Elhadad
On the safety of machine learning: Cyber-physical systems, decision sciences, and data products [pdf] Big data, 2016
Kush R. Varshney, Homa Alemzadeh
Tesla fatal crash: 'autopilot' mode sped up car before driver killed, report finds [pdf] The Guardian, 2018
Sam Levin
Challenges in autonomous vehicle validation: Keynote presentation abstract [pdf] Proceedings of the 1st International Workshop on Safe Control of Connected and Autonomous Vehicles, 2017
Philip Koopman
Developing artificial neural networks for safety critical systems [pdf] Neural Computing and Applications, 2007
Zeshan Kurd, Tim Kelly, Jim Austin
Google reports self-driving car mistakes: 272 failures and 13 near misses [pdf] The Guardian, 2016
Mark Harris
How Uber's Self-Driving Technology Could Have Failed In The Fatal Tempe Crash [pdf] Forbes, 2018
Jim McPherson
Debugging Machine Learning Tasks [pdf] arXiv preprint , 2018
Aleksandar Chakarov, Aditya Nori, Sriram Rajamani, Shayak Sen, Deepak Vijaykeerthy
On Human Intellect and Machine Failures: Troubleshooting Integrative Machine Learning Systems [pdf] AAAI, 2017
BBesmira Nushi, Ece Kamar, Eric Horvitz, Donald Kossmann
A Fault-Value Injection Approach for Multiple-Weight-Fault Tolerance of MNNs [pdf] Proceedings of the IEEE-INNS-ENNS, 2000
Itsuo Takanami, Masaru Sato, Yun Ping Yang
An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software [pdf] CoRR, 2017
Rick Salay, Rodrigo Queiroz, Krzysztof Czarnecki
Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks [pdf] CAV, 2017
Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer

Data Sources for Training Autonomous Driving Systems

NuScenes data collection [pdf] 2018
Computer vision for autonomous vehicles: Problems, datasets and state-of-the-art [pdf] 2014
Joel Janai, Fatma Güney, Aseem Behl, Andreas Geiger
When to use what data set for your self-driving car algorithm: An overview of publicly available driving datasets [pdf] IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), 2017
Hang Yin, Christian Berger
BDD100K: A Diverse Driving Video Database with Scalable Annotation Tooling [pdf] CoRR, 2018
Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, Trevor Darrell
Berkeley DeepDrive data collection [pdf] 2018
A Multi-sensor Traffic Scene Dataset with Omnidirectional Video [pdf] Ground Truth - What is a good dataset? CVPR Workshop, 2013
P. Koschorrek, T. Piccini, P. berg, M. Felsberg, L. Nielsen, R. Mester
MIT Autonomous Vehicle Technology Study: Large-Scale Deep Learning Based Analysis of Driver Behavior and Interaction with Automation [pdf] CoRR, 2017
Lex Fridman, Daniel E. Brown, Michael Glazer, William Angell, Spencer Dodd, Benedikt Jenik, Jack Terwilliger, Julia Kindelsberger, Li Ding, Sean Seaman, Hillary Abraham, Alea Mehler, Andrew Sipperley, Anthony Pettinato, Linda Angell, Bruce Mehler, Bryan Reimer
Ford campus vision and lidar data set [pdf] International Journal of Robotics Research, 2011
Gaurav Pandey, James R. McBride, Ryan Michael Eustice
Vision meets robotics: the KITTI dataset [pdf] The International Journal of Robotics Research, 2013
A. Geiger, P. Lenz, C. Stiller, R. Urtasun
Udacity data collection [pdf] 2018
Cityscapes data collection [pdf] 2018
The Oxford data collection [pdf] 2018
Semantic object classes in video: A high-definition ground truth database [pdf] Pattern Recognition Letters, 2009
Gabriel J. Brostow, Julien Fauqueur, Roberto Cipolla
The cambridge-driving labeled video database [pdf] 2018
Daimler pedestrian database [pdf] 2018
Caltech pedestrian database [pdf] 2018
Pyramid scene parsing network [pdf] IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, Jiaya Jia
Sgn: Sequential grouping networks for instance segmentation [pdf] 2017
Shu Liu, Jiaya Jia, Sanja Fidler, Raquel Urtasun
Segmentation and Recognition using Structure from Motion Point Clouds [pdf] ECCV, 2008
Gabriel J. Brostow, Jamie Shotton, Julien Fauqueur, Roberto Cipolla
Daimler Pedestrian Segmentation Benchmark Dataset [pdf] Proc. of the British Machine Vision Conference, 2013
F. Flohr, D. M. Gavrila.
A new benchmark for vision-based cyclist detection [pdf] IEEE Intelligent Vehicles Symposium (IV), 2016
X. Li, F. Flohr, Y. Yang, H. Xiong, M. Braun, S. Pan, K. Li, D. M. Gavrila
Pedestrian detection: A benchmark [pdf] IEEE Conference on Computer Vision and Pattern Recognition, 2009
P. Dollar, C. Wojek, B. Schiele, and P. Perona
Velodyne LiDAR for data collection [pdf] 2018
Sick LiDAR for data collection [pdf] 2018

Computational Hardware and Deployment

NVIDIA AI car computer Drive PX [pdf]
Tegra X2 [pdf]
DenverCore [pdf]
Pascal microarchitecture [pdf]
NVIDIA Drive AGX [pdf]
NVIDIA Volta [pdf]
R-CarV3H [pdf]
R-CarH3 [pdf]
Can fpgas beat gpus in accelerating next-generation deep neural networks? [pdf] Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2017
E. Nurvitadhi, G. Venkatesh, J. Sim, D. Marr, R. Huang, J. Ong Gee Hock, Y. T. Liew, K. Srivatsan, D. Moss, S. Subhaschandra, G. Boudoukh
Accelerating Deep Convolutional Neural Networks Using Specialized Hardware [pdf] 2015
Kalin Ovtcharov, Olatunji Ruwase, Joo-Young Kim, Jeremy Fowers, Karin Strauss, Eric S. Chung
Understanding performance differences of fpgas and gpus: (abtract only) [pdf] Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2018
Jason Cong, Zhenman Fang, Michael Lo, Hanrui Wang, Jingxian Xu, Shaochong Zhang