Automated Driving Systems (ADS) open up a new domain for the automotive industry and offer new possibilities for future transportation with higher efficiency and comfortable experiences. However, perception and sensing for autonomous driving under adverse weather conditions have been the problem that keeps autonomous vehicles (AVs) from going to higher autonomy for a long time. This paper assesses the influences and challenges that weather brings to ADS sensors in a systematic way, and surveys the solutions against inclement weather conditions. State-of-the-art algorithms and deep learning methods on perception enhancement with regard to each kind of weather, weather status classification, and remote sensing are thoroughly reported. Sensor fusion solutions, weather conditions coverage in currently available datasets, simulators, and experimental facilities are categorized. Additionally, potential ADS sensor candidates and developing research directions such as V2X (Vehicle to Everything) technologies are discussed. By looking into all kinds of major weather problems, and reviewing both sensor and computer science solutions in recent years, this survey points out the main moving trends of adverse weather problems in perception and sensing, i.e., advanced sensor fusion and more sophisticated machine learning techniques; and also the limitations brought by emerging 1550 nm LiDARs. In general, this work contributes a holistic overview of the obstacles and directions of perception and sensing research development in terms of adverse weather conditions. 

Perception and sensing for autonomous vehicles under adverse weather conditions: A survey

Reliable point cloud data is essential for perception tasks e.g. in robotics and autonomous driving applications. Adverse weather causes a specific type of noise to light detection and ranging (LiDAR) sensor data, which degrades the quality of the point clouds significantly. To address this issue, this letter presents a novel point cloud adverse weather denoising deep learning algorithm (4DenoiseNet). Our algorithm takes advantage of the time dimension unlike deep learning adverse weather denoising methods in the literature. It performs about 10% better in terms of intersection over union metric compared to the previous work and is more computationally efficient. These results are achieved on our novel SnowyKITTI dataset, which has over 40000 adverse weather annotated point clouds. Moreover, strong qualitative results on the Canadian Adverse Driving Conditions dataset indicate good generalizability to domain shifts and to different sensor intrinsics. 

4DenoiseNet: Adverse Weather Denoising From Adjacent Point Clouds

Reliable point cloud data is essential for perception tasks e.g. in robotics and autonomous driving applications. Adverse weather causes a specific type of noise to light detection and ranging (LiDAR) sensor data, which degrades the quality of the point clouds significantly. To address this issue, this letter presents a novel point cloud adverse weather denoising deep learning algorithm (4DenoiseNet). Our algorithm takes advantage of the time dimension unlike deep learning adverse weather denoising methods in the literature. It performs about 10% better in terms of intersection over union metric compared to the previous work and is more computationally efficient. These results are achieved on our novel SnowyKITTI dataset, which has over 40000 adverse weather annotated point clouds. Moreover, strong qualitative results on the Canadian Adverse Driving Conditions dataset indicate good generalizability to domain shifts and to different sensor intrinsics.

With the rapid advances in autonomous driving, it becomes critical to equip its sensing system with more holistic 3D perception. However, widely explored tasks like 3D detection or point cloud semantic segmentation focus on parsing either the objects (e.g. cars and pedestrians) or scenes (e.g. trees and buildings). In this work, we propose to address the challenging task of LiDAR-based Panoptic Segmentation, which aims to parse both objects and scenes in a unified manner. In particular, we propose Dynamic Shifting Network (DS-Net), which serves as an effective panoptic segmentation framework in the point cloud realm. DS-Net features a dynamic shifting module for complex LiDAR point cloud distributions. We observe that commonly used clustering algorithms like BFS or DBSCAN are incapable of handling complex autonomous driving scenes with non-uniform point cloud distributions and varying instance sizes. Thus, we present an efficient learnable clustering module, dynamic shifting, which adapts kernel functions on the fly for different instances. To further explore the temporal information, we extend the single-scan processing framework to its temporal version, namely 4D-DS-Net, for the task of 4D Panoptic Segmentation, where the same instance across multiple frames should be given the same ID prediction. Instead of naïvely appending a tracking module to DS-Net, we propose to solve the 4D panoptic segmentation in a more unified way. Specifically, 4D-DS-Net first constructs 4D data volume by aligning consecutive LiDAR scans, upon which the temporally unified instance clustering is performed to obtain the final results. Extensive experiments on two large-scale autonomous driving LiDAR datasets, SemanticKITTI and Panoptic nuScenes, are conducted to demonstrate the effectiveness and superior performance of the proposed solution. The code is publicly available at https://github.com/hongfz16/DS-Net.

Unified 3D and 4D Panoptic Segmentation via Dynamic Shifting Networks.

Autonomous vehicles rely on a variety of sensors to gather information about their surrounding. The vehicle’s behavior is planned based on the environment perception, making its reliability crucial for safety reasons. The active LiDAR sensor is able to create an accurate 3D representation of a scene, making it a valuable addition for environment perception for autonomous vehicles. Due to light scattering and occlusion, the LiDAR’s performance change under adverse weather conditions like fog, snow or rain. This limitation recently fostered a large body of research on approaches to alleviate the decrease in perception performance. In this survey, we gathered, analyzed, and discussed different aspects on dealing with adverse weather conditions in LiDAR-based environment perception. We address topics such as the availability of appropriate data, raw point cloud processing and denoising, robust perception algorithms and sensor fusion to mitigate adverse weather induced shortcomings. We furthermore identify the most pressing gaps in the current literature and pinpoint promising research directions. 

pdf/survey-on-lidar-perception-in-adverse-weather-conditions-2bpexzietz.pdf

Survey on LiDAR Perception in Adverse Weather Conditions

Accurate and efficient environmental awareness is a fundamental capability of autonomous driving technology and the real-time data collected by sensors offer autonomous vehicles an intuitive impression of their environment. Unfortunately, the ambient noise caused by varying weather conditions immediately affects the ability of autonomous vehicles to accurately understand their environment and its expected impact. In recent years, researchers have improved the environmental perception capabilities of simultaneous localization and mapping (SLAM), object detection and tracking, semantic segmentation and panoptic segmentation, but relatively few studies have focused on enhancing environmental perception capabilities in adverse weather conditions, such as rain, snow and fog. To enhance the environmental perception of autonomous vehicles in adverse weather, we developed a dynamic filtering method called Dynamic Distance–Intensity Outlier Removal (DDIOR), which integrates the distance and intensity of points based on the systematic and accurate analysis of LiDAR point cloud data characteristics in snowy weather. Experiments on the publicly available WADS dataset (Winter Adverse Driving dataSet) showed that our method can efficiently remove snow noise while fully preserving the detailed features of the environment.

/pdf/a-scalable-and-accurate-de-snowing-algorithm-for-lidar-point-3q4dg7lg.pdf

A Scalable and Accurate De-Snowing Algorithm for LiDAR Point Clouds in Winter

Fast and accurate semantic scene understanding is essential for mobile robots to operate in complex environments. An emerging research topic, panoptic segmentation, serves such a purpose by performing the tasks of semantic segmentation and instance segmentation in a unified framework. To improve the performance of LiDAR-based real-time panoptic segmentation, this study proposes a spatiotemporal sequential data fusion strategy that fused points in “thing classes” based on accurate data statistics. The data fusion strategy could increase the proportion of valuable data in unbalanced datasets, and thus managed to mitigate the adverse impact of class imbalance in the limited training data. Subsequently, by improving the codec network, the multiscale features shared by semantic and instance branches were efficiently aggregated to achieve accurate panoptic segmentation for each LiDAR scan. Experiments on the publicly available dataset SemanticKITTI showed that our approach could achieve an effective balance between accuracy and efficiency, and it was also applicable to other point cloud segmentation tasks.

https://www.mdpi.com/2072-4292/14/8/1775/pdf?version=1649907591

LiDAR-Based Real-Time Panoptic Segmentation via Spatiotemporal Sequential Data Fusion

Denoising is always an important step in seismic processing, in order to produce high-quality data for subsequent imaging and inversion. Different types of noise can be suppressed using targeted denoising methods. For outlier noise with singular amplitudes, many classical denoising methods suffer from signal leakage. To mitigate this issue, we developed a statistics-based mask method and incorporated it into the compressive sensing (CS) framework, in order to remove outlier noise. A statistical analysis for seismic data amplitudes was first used to identify the locations of traces containing outlier noise. Then, the outlier trace locations were compared with a mask matrix generated by jitter sampling, and we replaced the sampled traces of the jitter mask that had the outlier noise with their nearby unsampled traces. The optimized sampling matrix enabled us to effectively identify and remove outliers. This optimized mask strategy converts an outlier denoising problem into a data reconstruction problem. Finally, a sparsely constrained inverse problem was solved using a soft-threshold iteration solver to recover signals at the null locations. The feasibility and adaptability of the proposed method were demonstrated through numerical experiments for synthetic and field data. The results showed that the proposed method outperformed the conventional f-x deconvolution and median filter method, and could accurately suppress outlier noise and recover missed expected signals.

/pdf/outlier-denoising-using-a-novel-statistics-based-mask-1sy8johg.pdf

Outlier Denoising Using a Novel Statistics-Based Mask Strategy for Compressive Sensing

CNNs’ strong inductive biases of locality and weight sharing provide powerful representation ability and data sample utilization efficiency. However, the weight sharing might smooth out the discrepancy between similar pixels, resulting in the wrong matching between left and right camera-image pair in thin structures region and repetitive texture region. In this paper, we propose a novel Heterogeneous Feature Fusion in MLP (HFF-MLP) for Stereo matching. It employs MLP structure and relaxes the weights sharing in the local spatial region. To this end, pixels in thin structures region and repetitive texture region are dealt with independently using the exclusive weights. Based on HFF -MLP module, we design a real-time network, i.e., MLP-Stereo. Experimental results show that our proposed HFF-MLP achieves competitive results on KITTI 2015 test dataset with the running time of 46 milliseconds. Furthermore, it performs much better than other real-time network in thin structures region and repetitive texture region.

MLP-Stereo: Heterogeneous Feature Fusion in MLP for Stereo Matching

Although CNN has powerful learning capability, it is still difficult for CNN to judge corresponding points in the occlusion region. The ghosting effect in the occlusion region during feature warping is the bottleneck of performance improvement for many stereo matching networks. In this paper, we propose an Occlusion-Aware Refinement Module (OARM), which can learn a rough occlusion map from multi-scale aggregated cost volumes without occlusion supervision to mask and filter pernicious occluded regions in a warped image. Cooperating with well-designed simple yet efficient 2D-based Intra/Cross-Level Aggregation Modules to effectively and efficiently aggregate information of different scales before the disparity refinement stage, OARM helps our proposed OM-Net achieve an error rate of 1.82% (D1-all) on KITTI 2015 dataset, which is even better than most 3D-based networks. Meanwhile, OMNet keeps real-time characteristic and could process a 1248×384 resolution image pair at 26 fps.

OMNET: Real-Time Stereo Matching with Unsupervised Occlusion Mask

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Weiqi Wang

Author Tools

Chat about Author