ABSTRACT Point cloud registration is the process of aligning and merging multiple point clouds into a same coordinate system. It has many applications in computer vision, robotics, 3D reconstruction, augmented reality, and autonomous driving. Despite its wide-ranging utility, traditional algorithms grapple with challenges such as substantial noise, outliers, limited overlap, motion distortions, and handling large scenes. These complexities impede the effective deployment of point cloud registration in practical real-world scenarios. This paper conducts a comprehensive inquiry into the domain of deep learning-based point cloud registration. It synthesizes recent advancements in algorithms about point cloud registration grounded in deep learning. The exposition delineates insights from four distinct perspectives: multi-layer perceptron, deep neural network, graph convolutional network, and attention mechanism. Furthermore, the paper systematically reviews prevalent datasets and evaluation metrics associated with deep learning-based point cloud registration. A discussion on the existing challenges in point cloud registration and a forward-looking exploration of potential research directions are included.

Deep learning-based point cloud registration: a comprehensive investigation

Despite the fact that point cloud registration under noisy conditions has recently begun to be tackled by several non-correspondence algorithms, they neither struggle to fuse the global features nor abandon early state estimation during the iterative alignment. To solve the problem, we propose a novel method named R-PCR (recurrent point cloud registration). R-PCR employs a lightweight cross-concatenation module and large receptive network to improve global feature performance. More importantly, it treats the point registration procedure as a high-order Markov decision process and introduces a recurrent neural network for end-to-end optimization. The experiments on indoor and outdoor benchmarks show that R-PCR outperforms state-of-the-art counterparts. The mean average error of rotation and translation of the aligned point cloud pairs are, respectively, reduced by 75% and 66% on the indoor benchmark (ScanObjectNN), and simultaneously by 50% and 37.5% on the outdoor benchmark (AirLoc).

https://www.mdpi.com/2072-4292/15/7/1889/pdf?version=1680773282

R-PCR: Recurrent Point Cloud Registration Using High-Order Markov Decision

While visual localization has made significant advances in recent years, it still lacks robustness in low-light situations. Thermal camera images, which capture temperature data, provide a potential solution for these environments. However, the scarcity of well-annotated, publicly available datasets for thermal localization, particularly those focused on absolute pose estimation, impedes further advancement in this field. In this research, we introduce a novel dataset that includes six-degree-of-freedom (6-DoF) absolute poses of query images for large-scale, realistic aerial localization of thermal images. Besides, we introduce a render-to-localization pipeline tailored for thermal image localization. This pipeline predicts the 6-DoF pose of a query using a synthetic technique based on geometric refined thermal model. Experimental results demonstrate the effectiveness of our method on this newly proposed dataset. Notably, our method achieves a median position error of less than 1.5 m and a median angle error of less than 1.5° under diverse test conditions. A comprehensive analysis of factors influencing localization accuracy is also provided. Our code and dataset will be available at https://github.com/RingoWRW/ATLoc.

ATLoc: Aerial Thermal Images Localization via View Synthesis

Thermography is especially valuable for the military and other users of surveillance cameras. Some recent methods based on Neural Radiance Fields (NeRF) are proposed to reconstruct the thermal scenes in 3D from a set of thermal and RGB images. However, unlike NeRF, 3D Gaussian splatting (3DGS) prevails due to its rapid training and real-time rendering. In this work, we propose ThermalGaussian, the first thermal 3DGS approach capable of rendering high-quality images in RGB and thermal modalities. We first calibrate the RGB camera and the thermal camera to ensure that both modalities are accurately aligned. Subsequently, we use the registered images to learn the multimodal 3D Gaussians. To prevent the overfitting of any single modality, we introduce several multimodal regularization constraints. We also develop smoothing constraints tailored to the physical characteristics of the thermal modality. Besides, we contribute a real-world dataset named RGBT-Scenes, captured by a hand-hold thermal-infrared camera, facilitating future research on thermal scene reconstruction. We conduct comprehensive experiments to show that ThermalGaussian achieves photorealistic rendering of thermal images and improves the rendering quality of RGB images. With the proposed multimodal regularization constraints, we also reduced the model's storage cost by 90\%. The code and dataset will be released.

ThermalGaussian: Thermal 3D Gaussian Splatting

As a prerequisite for many 3D visualization tasks, point cloud registration has a wide range of applications in 3D scene reconstruction, pose estimation, navigation, and remote sensing. However, due to the limited overlap of point clouds, the presence of noise and the incompleteness of the data, existing feature-based matching methods tend to produce higher outlier matches, thus reducing the quality of the registration. Therefore, the generation of reliable feature descriptors and the filtering of outliers become the key to solving these problems. To this end, we propose a multi-local-to-global registration (MLGR) method. First, in order to obtain reliable correspondences, we design a simple but effective network module named the local geometric network (LG-Net), which can generate discriminative feature descriptors to reduce the outlier matches by learning the local latent geometric information of the point cloud. In addition, we propose a multi-local-to-global registration strategy to further filter outlier matches. We compute the hypothetical transformation matrix from local patch matches. The point match evaluated as an inlier under multiple hypothetical transformations will receive a higher score, and low-scoring point matches will be rejected. Finally, our method is quite robust under different numbers of samples, as it does not require sampling a large number of correspondences to boost the performance. The numerous experiments on well-known public datasets, including KITTI, 3DMatch, and ModelNet, have proven the effectiveness and robustness of our method. Compared with the state of the art, our method has the lowest relative rotation error and relative translation error on the KITTI, and consistently leads in feature matching recall, inlier ratio, and registration recall on 3DMatch under different numbers of point correspondences, which proves the robustness of our method. In particular, the inlier ratio is significantly improved by 3.62% and 4.36% on 3DMatch and 3DLoMatch, respectively. In general, the performance of our method is more superior and robust than the current state of the art.

A Robust Multi-Local to Global with Outlier Filtering for Point Cloud Registration

Despite the significant progress in 6-DoF visual localization, researchers are mostly driven by ground-level benchmarks. Compared with aerial oblique photography, ground-level map collection lacks scalability and complete coverage. In this work, we propose to go beyond the traditional ground-level setting and exploit the cross-view localization from aerial to ground. We solve this problem by formulating camera pose estimation as an iterative render-and-compare pipeline and enhancing the robustness through augmenting seeds from noisy initial priors. As no public dataset exists for the studied problem, we collect a new dataset that provides a variety of cross-view images from smartphones and drones and develop a semi-automatic system to acquire ground-truth poses for query images. We benchmark our method as well as several state-of-the-art baselines and demonstrate that our method outperforms other approaches by a large margin.

pdf/render-and-compare-cross-view-6-dof-localization-from-noisy-wgflzljb.pdf

Render-and-Compare: Cross-View 6 DoF Localization from Noisy Prior

Despite significant progress in global localization of Unmanned Aerial Vehicles (UAVs) in GPS-denied environments, existing methods remain constrained by the availability of datasets. Current datasets often focus on small-scale scenes and lack viewpoint variability, accurate ground truth (GT) pose, and UAV build-in sensor data. To address these limitations, we introduce a large-scale 6-DoF UAV dataset for localization (UAVD4L) and develop a two-stage 6-DoF localization pipeline (UAVLoc), which consists of offline synthetic data generation and online visual localization. Additionally, based on the 6-DoF estimator, we design a hierarchical system for tracking ground target in 3D space. Experimental results on the new dataset demonstrate the effectiveness of the proposed approach. Code and dataset are available at https://github.com/RingoWRW/UAVD4L

UAVD4L: A Large-Scale Dataset for UAV 6-DoF Localization

Despite the impressive progress in visual localisation, 6‐DoF cross‐view localisation is still a challenging task in the computer vision community due to the huge appearance changes. To address this issue, the authors propose the CLaSP, a coarse‐to‐fine framework, which leverages a synthetic panorama to facilitate cross‐view 6‐DoF localisation in a large‐scale scene. The authors first leverage a segmentation map to correct the prior pose, followed by a synthetic panorama on the ground to enable coarse pose estimation combined with a template matching method. The authors finally formulate the refine localisation process as feature matching and pose refinement to obtain the final result. The authors evaluate the performance of the CLaSP and several state‐of‐the‐art baselines on the Airloc dataset, which demonstrates the effectiveness of our proposed framework.

CLaSP: Cross‐view 6‐DoF localisation assisted by synthetic panorama

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Xiaoya Cheng

Author Tools

Chat about Author