nuScenes: A Multimodal Dataset for Autonomous Driving
Holger Caesar,Varun Bankiti,Alex H. Lang,Sourabh Vora,Venice Erin Liong,Qiang Xu,Anush Krishnan,Yu Pan,Giancarlo Baldan,Oscar Beijbom +9 more
- 14 Jun 2020
- pp 11621-11631
TL;DR: nuScenes as discussed by the authors is the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view.
read more
Abstract: Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Learning-Based Extended Object Tracking Using Hierarchical Truncation Measurement Model With Automotive Radar
Yuxuan Xia,Pu Wang,Karl Berntorp,Lennart Svensson,Karl Granstrom,Hassan Mansour,Petros T. Boufounos,Philip Orlik +7 more
TL;DR: This paper presents a data-driven measurement model for extended object tracking (EOT) with automotive radar that is modeled as a hierarchical truncated Gaussian (HTG) with structural geometry parameters that can be learned from the training data.
38
Real-Time 3D Single Object Tracking With Transformer
01 Jan 2023
TL;DR: PTT-Net as discussed by the authors uses the powerful transformer architecture to generate fine-tuned attention features by computing attention weights, which guides the tracker focusing on the important features of the target and improves the tracking ability in complex scenarios.
Delving Into the Devils of Bird's-Eye-View Perception: A Review, Evaluation and Recipe
Hongyang Li,Chonghao Sima,Jifeng Dai,Wenhai Wang,Lewei Lu,H. Wang,Jia Zeng,Zhiqi Li,Jiazhi Yang,Hanming Deng,Hanqin Tian,Enze Xie,Jiangwei Xie,Li Chen,Tianyu Li,Yang Li,Yulu Gao,Xiaosong Jia,Si Liu,Junming Shi,Dahua Lin,Yu Qiao +21 more
TL;DR: BEV perception is gaining traction due to its intuitive representation and fusion-friendliness. Core challenges include reconstruction of 3D information, ground truth annotation, pipeline formulation, and adaptation to sensor variations. This survey reviews recent works, analyzes solutions, and introduces practical guidelines for improving BEV perception performance.
38
Out-of-Distribution Detection for Automotive Perception
Julia Nitsch,Masha Itkina,Ransalu Senanayake,Juan Nieto,Max Schmidt,Roland Siegwart,Mykel J. Kochenderfer,Cesar Cadena +7 more
- 19 Sep 2021
TL;DR: In this article, a method for determining whether inputs are OOD is presented, which does not require OOD data during training and does not increase the computational cost of inference, which is important for automotive applications with limited computational resources and real-time constraints.
38
A Scalable and Accurate De-Snowing Algorithm for LiDAR Point Clouds in Winter
TL;DR: Li et al. as discussed by the authors developed a dynamic filtering method called Dynamic Distance-Intensity Outlier Removal (DDIOR), which integrates the distance and intensity of points based on the systematic and accurate analysis of LiDAR point cloud data characteristics in snowy weather.
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Proceedings Article
ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky,Ilya Sutskever,Geoffrey E. Hinton +2 more
- 03 Dec 2012
TL;DR: The state-of-the-art performance of CNNs was achieved by Deep Convolutional Neural Networks (DCNNs) as discussed by the authors, which consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax.
ImageNet: A large-scale hierarchical image database
Jia Deng,Wei Dong,Richard Socher,Li-Jia Li,Kai Li,Li Fei-Fei +5 more
- 20 Jun 2009
TL;DR: A new database called “ImageNet” is introduced, a large-scale ontology of images built upon the backbone of the WordNet structure, much larger in scale and diversity and much more accurate than the current image datasets.
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
TL;DR: This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features.
Histograms of oriented gradients for human detection
Navneet Dalal,Bill Triggs +1 more
- 20 Jun 2005
TL;DR: It is shown experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection, and the influence of each stage of the computation on performance is studied.