Abstract: The goal of our work is to provide a fast and accurate method to estimate the camera motion from RGB-D images. Our approach registers two consecutive RGB-D frames directly upon each other by minimizing the photometric error. We estimate the camera motion using non-linear minimization in combination with a coarse-to-fine scheme. To allow for noise and outliers in the image data, we propose to use a robust error function that reduces the influence of large residuals. Furthermore, our formulation allows for the inclusion of a motion model which can be based on prior knowledge, temporal filtering, or additional sensors like an IMU. Our method is attractive for robots with limited computational resources as it runs in real-time on a single CPU core and has a small, constant memory footprint. In an extensive set of experiments carried out both on a benchmark dataset and synthetic data, we demonstrate that our approach is more accurate and robust than previous methods. We provide our software under an open source license.

Abstract: We propose a fundamentally novel approach to real-time visual odometry for a monocular camera. It allows to benefit from the simplicity and accuracy of dense tracking - which does not depend on visual features - while running in real-time on a CPU. The key idea is to continuously estimate a semi-dense inverse depth map for the current frame, which in turn is used to track the motion of the camera using dense image alignment. More specifically, we estimate the depth of all pixels which have a non-negligible image gradient. Each estimate is represented as a Gaussian probability distribution over the inverse depth. We propagate this information over time, and update it with new measurements as new images arrive. In terms of tracking accuracy and computational speed, the proposed method compares favorably to both state-of-the-art dense and feature-based visual odometry and SLAM algorithms. As our method runs in real-time on a CPU, it is of large practical value for robotics and augmented reality applications.

Abstract: This paper describes extensions to the Kintinuous [1] algorithm for spatially extended KinectFusion, incorporating the following additions: (i) the integration of multiple 6DOF camera odometry estimation methods for robust tracking; (ii) a novel GPU-based implementation of an existing dense RGB-D visual odometry algorithm; (iii) advanced fused realtime surface coloring. These extensions are validated with extensive experimental results, both quantitative and qualitative, demonstrating the ability to build dense fully colored models of spatially extended environments for robotics and virtual reality applications while remaining robust against scenes with challenging sets of geometric and visual features.

Abstract: Long-term autonomous mobile robot operation requires considering place recognition decisions with great caution. A single incorrect decision that is not detected and reconsidered can corrupt the environment model that the robot is trying to build and maintain. This work describes a consensus-based approach to robust place recognition over time, that takes into account all the available information to detect and remove past incorrect loop closures. The main novelties of our work are: (1) the ability of realizing that, in light of new evidence, an incorrect past loop closing decision has been made; the incorrect information can be removed thus recovering the correct estimation with a novel algorithm; (2) extending our proposal to incremental operation; and (3) handling multi-session, spatially related or unrelated scenarios in a unified manner. We demonstrate our proposal, the RRR algorithm, on different odometry systems, e.g. visual or laser, using different front-end loop-closing techniques. For our experiments we use the efficient graph optimization framework g2o as back-end. We back our claims up with several experiments carried out on real data, in single and multi-session experiments showing better results than those obtained by state-of-the-art methods, comparisons against whom are also presented.

Abstract: Multiple cameras are increasingly prevalent on robotic and human-driven vehicles. These cameras come in a variety of wide-angle, fish-eye, and catadioptric models. Furthermore, wheel odometry is generally available on the vehicles on which the cameras are mounted. For robustness, vision applications tend to use wheel odometry as a strong prior for camera pose estimation, and in these cases, an accurate extrinsic calibration is required in addition to an accurate intrinsic calibration. To date, there is no known work on automatic intrinsic calibration of generic cameras, and more importantly, automatic extrinsic calibration of a rig with multiple generic cameras and odometry. We propose an easy-to-use automated pipeline that handles both intrinsic and extrinsic calibration; we do not assume that there are overlapping fields of view. At the begining, we run an intrinsic calibration for each generic camera. The intrinsic calibration is automatic and requires a chessboard. Subsequently, we run an extrinsic calibration which finds all camera-odometry transforms. The extrinsic calibration is unsupervised, uses natural features, and only requires the vehicle to be driven around for a short time. The intrinsic parameters are optimized in a final bundle adjustment step in the extrinsic calibration. In addition, the pipeline produces a globally-consistent sparse map of landmarks which can be used for visual localization. The pipeline is publicly available as a standalone C++ package.

Abstract: We introduce our new quadrotor platform for realizing autonomous navigation in unknown indoor/outdoor environments. Autonomous waypoint navigation, obstacle avoidance and flight control is implemented on-board. The system does not require a special environment, artificial markers or an external reference system. We developed a monolithic, mechanically damped perception unit which is equipped with a stereo camera pair, an Inertial Measurement Unit (IMU), two processor-and an FPGA board. Stereo images are processed on the FPGA by the Semi-Global Matching algorithm. Keyframe-based stereo odometry is fused with IMU data compensating for time delays that are induced by the vision pipeline. The system state estimate is used for control and on-board 3D mapping. An operator can set waypoints in the map, while the quadrotor autonomously plans its path avoiding obstacles. We show experiments with the quadrotor flying from inside a building to the outside and vice versa, traversing a window and a door respectively. A video of the experiments is part of this work. To the best of our knowledge, this is the first autonomously flying system with complete on-board processing that performs waypoint navigation with obstacle avoidance in geometrically unconstrained, complex indoor/outdoor environments.

Abstract: This paper describes flow-relative and flow-aided navigation of a biomimetic underwater vehicle using an artificial lateral line for flow sensing. Most of the aquatic animals have flow sensing organs, but there are no man-made analogues to those sensors currently in use on underwater vehicles. Here, we show that artificial lateral line sensing can be used for detecting hydrodynamic regimens and for controlling the robot’s motion with respect to the flow. We implement station holding of an underwater vehicle in a steady stream and in the wake of a bluff object. We show that lateral line sensing can provide a speed estimate of an underwater robot thus functioning as a short-term odometry for robot navigation. We also demonstrate navigation with respect to the flow in periodic turbulence and show that controlling the position of the robot in the reduced flow zone in the wake of an object reduces a vehicle’s energy consumption.

Abstract: Consider a differential-drive mobile robot equipped with an on-board exteroceptive sensor that can estimate its own motion, e.g., a range-finder. Calibration of this robot involves estimating six parameters: three for the odometry (radii and distance between the wheels) and three for the pose of the sensor with respect to the robot. After analyzing the observability of this problem, this paper describes a method for calibrating all parameters at the same time, without the need for external sensors or devices, using only the measurement of the wheel velocities and the data from the exteroceptive sensor. The method does not require the robot to move along particular trajectories. Simultaneous calibration is formulated as a maximum-likelihood problem and the solution is found in a closed form. Experimental results show that the accuracy of the proposed calibration method is very close to the attainable limit given by the Cramer–Rao bound.

Abstract: When measurements from multiple sensors are combined for real-time motion estimation, the time instant at which each measurement was recorded must be precisely known. In practice, however, the timestamps of each sensor's measurements are typically affected by a delay, which is different for each sensor. This gives rise to a temporal misalignment (i.e., a time offset) between the sensors' data streams. In this work, we propose an online approach for estimating the time offset between the data obtained from different sensors. Specifically, we focus on the problem of motion estimation using visual and inertial sensors in extended Kalman filter (EKF)-based methods. The key idea proposed here is to explicitly include the time offset between the camera and IMU in the EKF state vector, and estimate it online along with all other variables of interest (the IMU pose, the camera-to-IMU calibration, etc). Our proposed approach is general, and can be employed in several classes of estimation problems, such as motion estimation based on mapped features, EKF-based SLAM, or visual-inertial odometry. Our simulation and experimental results demonstrate that the proposed approach yields high-precision, consistent estimates, in scenarios involving both constant and time-varying offsets.

Abstract: This paper presents a precise localization algorithm for vehicles in 3D urban environment with only one 2D LIDAR and odometry information. A novel idea of synthetic 2D LIDAR is proposed to solve the localization problem on a virtual 2D plane. A Monte Carlo Localization scheme is adopted for vehicle position estimation, based on synthetic LIDAR measurements and odometry information. The accuracy and robustness of the proposed algorithm are demonstrated by performing real time localization in a 1.5 km driving test around the NUS campus area.

Abstract: An accurate dynamic ultrasonic hybrid localization system is presented for autonomous navigation of indoor mobile robots using multiple ultrasonic distance measurements and an extended Kalman filter (EKF). The ultrasonic sensor subsystem is composed of several ultrasonic transmitters (Txs) attached to the ceiling at known positions and several ultrasonic receivers equilaterally located on the top of the mobile robot, which has a moving speed that is not negligible. An EKF-based algorithm with a state/observation vector composed of the robot pose (or the position and the orientation) is presented using odometric and ultrasonic distance measurements. A dynamic distance estimation method is proposed to track the estimates of ultrasonic distance information from available Txs of interest using both odometric information from the robot and actual ultrasonic distance measurements. This continuous dynamic distance estimation allows persistent use of the hybrid self-localization algorithm to accurately determine the pose of the robot. The experimental results with various trajectories clearly show that the proposed method is much more accurate than only the hybrid self-localization algorithm (without the dynamic distance estimation method).

Abstract: The magnetic field in indoor environments is rich in features and exceptionally easy to sense. In conjunction with a suitable form of odometry, such as signals produced from inertial sensors or wheel encoders, a map of this field can be used to precisely localize a human or robot in an indoor environment. We show how the use of this field yields significant improvements in terms of localization accuracy for both legged and non-legged locomotion. We suggest various likelihood functions for sequential Monte Carlo localization and evaluate their performance based on magnetic maps of different resolutions. Specifically, we investigate the influence that measurement representation (e.g., intensity-based, vector-based) and map resolution have on localization accuracy, robustness, and complexity. Compared to other localization approaches (e.g., camera-based, LIDAR-based), there exist far fever privacy concerns when sensing the indoor environment's magnetic field. Furthermore, the required sensors are less costly, compact, and have a lower raw data rate and power consumption. The combination of technical and privacy-related advantages makes the use of the magnetic field a very viable solution to indoor navigation for both humans and robots.

Abstract: We present a Simultaneous Localization and Mapping (SLAM) algorithm based on measurements of the ambient magnetic field strength (MagSLAM) that allows quasi-real-time mapping and localization in buildings, where pedestrians with foot-mounted sensors are the subjects to be localized. We assume two components to be present: firstly a source of odometry (human step measurements), and secondly a sensor of the local magnetic field intensity. Our implementation follows the FastSLAM factorization using a particle filter. We augment the hexagonal transition map used in the pre-existing FootSLAM algorithm with local maps of the magnetic field strength, binned in a hierarchical hexagonal structure. We performed extensive experiments in a number of different buildings and present the results for five data sets for which we have ground truth location information. We consider the results obtained using MagSLAM to be strong evidence that scalable and accurate localization is possible without an a priori map.

Abstract: Array beamforming techniques allow for the generation of 3-D spatial filters which can be used to localize objects in a large field of view (FOV) without the need for mechanical scanning. By combining broadband beamforming with a sparse, random array of receivers, we have constructed a low-cost, yet powerful, in-air sonar system, which is suited for a wide range of robotic applications. Experimental results in unmodified office environments show the performance of the sonar sensor. In particular, we document the sensor's capacity to produce 3-D location measurements in the presence of multiple highly overlapping echoes. We show how this capability makes possible the combination of a wide FOV with accurate 3-D localization, allowing the sensor to operate under real-time constraints in realistic environments. To demonstrate the use of this sensor, we describe an odometry application that estimates egomotion of a mobile robot using acoustic flow.

Abstract: We present a robust plane finding algorithm that when combined with plane-based frame-to-frame registration gives accurate real-time pose estimation. Our plane extraction is capable of handling large and sparse datasets such as those generated from spinning multi-laser sensors such as the Velodyne HDL-32E LiDAR. We test our algorithm on frame-to-frame registration in a closed-loop indoor path comprising 827 successive 3D laser scans (over 57 million points), using no additional information (e.g., odometry, IMU). Our algorithm outperforms, in both accuracy and time, three state-of-the-art methods, based on iterative closest point (ICP), plane-based randomized Hough transform, and planar region growing.

Abstract: In this paper, we present Gaussian Process Gauss-Newton (GPGN), an algorithm for non-parametric, continuous-time, nonlinear, batch state estimation. This work adapts the methods of Gaussian process (GP) regression to address the problem of batch simultaneous localization and mapping (SLAM) by using the Gauss-Newton optimization method. In particular, we formulate the estimation problem with a continuous-time state model, along with the more conventional discrete-time measurements. Two derivations are presented in this paper, reflecting both the weight-space and function-space approaches from the GP regression literature. Validation is conducted through simulations and a hardware experiment, which utilizes the well-understood problem of two-dimensional SLAM as an illustrative example. The performance is compared with the traditional discrete-time batch Gauss-Newton approach, and we also show that GPGN can be employed to estimate motion with only range/bearing measurements of landmarks (i.e. no odometry), even when there are not enough measurements to constrain the pose at a given timestep.

Abstract: Text in an image provides vital information for interpreting its contents, and text in a scene can aide with a variety of tasks from navigation, to obstacle avoidance, and odometry. Despite its value, however, identifying general text in images remains a challenging research problem. Motivated by the need to consider the widely varying forms of natural text, we propose a bottom-up approach to the problem which reflects the `characterness' of an image region. In this sense our approach mirrors the move from saliency detection methods to measures of `objectness'. In order to measure the characterness we develop three novel cues that are tailored for character detection, and a Bayesian method for their integration. Because text is made up of sets of characters, we then design a Markov random field (MRF) model so as to exploit the inherent dependencies between characters. We experimentally demonstrate the effectiveness of our characterness cues as well as the advantage of Bayesian multi-cue integration. The proposed text detector outperforms state-of-the-art methods on a few benchmark scene text detection datasets. We also show that our measurement of `characterness' is superior than state-of-the-art saliency detection models when applied to the same task.

Abstract: Mobile laser scanning puts high requirements on the accuracy of the positioning systems and the calibration of the measurement system. We present a novel algorithmic approach for calibration with the goal of improving the measurement accuracy of mobile laser scanners. We describe a general framework for calibrating mobile sensor platforms that estimates all configuration parameters for any arrangement of positioning sensors, including odometry. In addition, we present a novel semi-rigid Simultaneous Localization and Mapping (SLAM) algorithm that corrects the vehicle position at every point in time along its trajectory, while simultaneously improving the quality and precision of the entire acquired point cloud. Using this algorithm, the temporary failure of accurate external positioning systems or the lack thereof can be compensated for. We demonstrate the capabilities of the two newly proposed algorithms on a wide variety of datasets.

Abstract: We present a highly effective approach for the calibration of vehicle models. The approach combines the output error technique of system identification theory and the convolution integral solutions of linear systems and stochastic calculus. Rather than calibrate the system differential equation directly for unknown parameters, we calibrate its first integral. This integrated prediction error minimization (IPEM) approach is advantageous because it requires only low-frequency observations of state, and produces unbiased parameter estimates that optimize simulation accuracy for the chosen time horizon. We address the calibration of models that describe both systematic and stochastic dynamics, such that uncertainties can be computed for model predictions. We resolve numerous implementation issues in the application of IPEM, such as the efficient linearization of the dynamics integral with respect to parameters, the treatment of uncertainty in initial conditions, and the formulation of stochastic measurements and measurement covariances. While the technique can be used for any dynamical system, we demonstrate its usefulness for the calibration of wheeled vehicle models used in control and estimation. Specifically we calibrate models of odometry, powertrain dynamics, and wheel slip as it affects body frame velocity. Experimental results are provided for a variety of indoor and outdoor platforms.

Abstract: The navigation of a miniature aerial vehicle (MAV) in GPS-denied environments requires a robust embedded visual localization method. In this paper, we describe a simple but efficient stereo visual odometry algorithm, called eVO, running onboard our quadricopter MAV at video-rate. The proposed eVO algorithm relies on a keyframe scheme which allows to decrease the estimation drift and to reduce the computational cost. We study quantitatively the influence of the main parameters of the algorithm and tune them for optimal performance on various datasets. The eVO algorithm has been submitted to the KITTI odometry benchmark [1] where it ranks first at the date of submission, with an average translational drift of 1.93% and an average angular drift of less than 0.076 degres/m. Besides, we have made several experiments with our MAV with egolocalization given by eVO, for instance for autonomous 3D environment modeling.

Abstract: Automation of logistics tasks for small lot sizes and flexible production processes requires intuitive and easy-to-use systems that allow non-expert shop floor workers to naturally instruct transportation systems. To this end, we present a novel laser-based scheme for teach-and-repeat of mobile robot trajectories that relies on scan matching to localize the robot relative to a taught trajectory, which is represented by a sequence of raw odometry and 2D laser data. This approach has two advantages. First, it does not require to build a globally consistent metrical map of the environment, which reduces setup time. Second, the direct use of raw sensor data avoids additional errors that might be introduced by the fact that grid maps only provide an approximation of the environment. Real-world experiments carried out with a holonomic and a differential drive platform demonstrate that our approach repeats trajectories with an accuracy of a few millimeters. A comparison with a standard Monte Carlo localization approach on grid maps furthermore reveals that our method yields lower tracking errors for teach-and-repeat tasks.

Abstract: In this paper, we present an integrated navigation system that allows humanoid robots to autonomously navigate in unknown, cluttered environments. From the data of an onboard consumer-grade depth camera, our system estimates the robot's pose to compensate for drift of odometry and maintains a heightmap representation of the environment. Based on this model, our system iteratively computes sequences of safe actions including footsteps and whole-body motions, leading the robot to target locations. Hereby, the planner chooses from a set of actions that consists of planar footsteps, step-over actions, as well as parameterized step-onto and step-down actions. To efficiently check for collisions during planning, we developed a new approach that takes into account the shape of the robot and the obstacles. As we demonstrate in experiments with a Nao humanoid, our system leads to robust navigation in cluttered environments and the robot is able to traverse highly challenging passages.

Abstract: In this article, a vision-based technique for obstacle avoidance and target identification is combined with haptic feedback to develop a new teleoperated navigation system for underactuated aerial vehicles in unknown environments. A three-dimensional (3-D) map of the surrounding environment is built by matching the keypoints among several images, which are acquired by an onboard camera and stored in a buffer together with the corresponding estimated odometry. Hence, based on the 3-D map, a visual identification algorithm is employed to localize both obstacles and the desired target to build a virtual field accordingly. A bilateral control system has been developed such that an operator can safely navigate in an unknown environment and perceive it by means of a vision-based haptic force-feedback device. Experimental tests in an indoor environment verify the effectiveness of the proposed teleoperated control.

Abstract: We propose an approach for autonomous task partitioning in swarms of foraging robots. Task partitioning is the process of decomposing tasks into sub-tasks. Task partitioning impacts tasks execution and associated costs. Our approach is characterized by the use of a cost function, mapping the size of sub-tasks to the overall task cost. The robots model the cost function and use the model to select sub-tasks to perform, aiming to minimize costs. Our approach separates the task partitioning process from task-specific actions and it does not require a priori assumptions to be made about the best partitioning strategy to employ. We study a foraging scenario in which object transportation is performed by different robots, each moving objects for a limited distance. The robots autonomously decide the distance traveled on the basis of our approach. The robots use odometry for navigational purposes; we show that task partitioning reduces the impact of odometry errors and improves performance. We validate our approach using simulation-based experiments. We study how the swarm partitions transportation under a number of experimental conditions characterized by different levels of odometry accuracy, size of the environment and the swarm, and total transportation distance. Our approach leads to partitioning solutions that are appropriate for each condition.

Abstract: The use of foot-mounted inertial measurement units (IMUs) has shown promising results in providing accurate human odometry as a component of accurate indoor pedestrian navigation. The specifications of these sensors, such as the sampling frequency have to meet requirements related to human motion.

Abstract: In this paper, we present SMART (Sequence Matching Across Route Traversals): a vision- based place recognition system that uses whole image matching techniques and odometry information to improve the precision-recall performance, latency and general applicability of the SeqSLAM algorithm. We evaluate the system’s performance on challenging day and night journeys over several kilometres at widely varying vehicle velocities from 0 to 60 km/h, compare performance to the current state-of- the-art SeqSLAM algorithm, and provide parameter studies that evaluate the effectiveness of each system component. Using 30-metre sequences, SMART achieves place recognition performance of 81% recall at 100% precision, outperforming SeqSLAM, and is robust to significant degradations in odometry.

Abstract: Precise localization of mobile robots in uncertain environments is a fundamental and crucial issue in robotics. In this paper, to deal with the unbounded accumulated errors of dead reckoning (DR)-based localization, wireless sensor network (WSN)-based localization is applied to calibrate the uncertainty of odometry using a Kalman filter (KF). In addition, to further aid in obtaining precise positions and reduce uncertainty, a novel backward dead reckoning (BDR) localization approach is proposed. The experimental results demonstrate the success and reliability of the proposed method.

Abstract: We present a 6-degree-of-freedom (6-DoF) pose localization method for a monocular camera in a 3D point-cloud dense map prebuilt by depth sensors (eg, RGB-D sensor, laser scanner, etc) We employ fast and robust 2D feature detection on the real camera to be matched against features from a virtual view The virtual view (color and depth images) is constructed by projecting the map's 3D points onto a plane using the previous localized pose of the real camera 2D-to-3D point correspondences are obtained from the inherent relationship between the real camera's 2D features and their matches on the virtual depth image (projected 3D points) Thus, we can solve the Perspective-n-Point (PnP) problem in order to find the relative pose between the real and virtual cameras With the help of RANSAC, the projection error is minimized even further Finally, the real camera's pose is solved with respect to the map by a simple frame transformation This procedure repeats for each time step (except for the initial case) Our results indicate that a monocular camera alone can be localized within the map in real-time (at QVGA-resolution) Our method differentiates from others in that no chain of poses is needed or kept Our localization is not susceptible to drift because the history of motion (odometry) is mostly independent over each PnP + RANSAC solution, which throws away past errors In fact, the previous known pose only acts as a region of interest to associate 2D features on the real image with 3D points in the map The applications of our proposed method are various, and perhaps it is a solution that has not been attempted before

Abstract: This paper reports on the use of planar patches as features in a real-time simultaneous localization and mapping (SLAM) system to model smooth surfaces as piecewise-planar. This approach works well for using observed point clouds to correct odometry error, even when the point cloud is sparse. Such sparse point clouds are easily derived by Doppler velocity log sensors for underwater navigation. Each planar patch contained in this point cloud can be constrained in a factor-graph-based approach to SLAM so that neighboring patches are sufficiently coplanar so as to constrain the robot trajectory, but not so much so that the curvature of the surface is lost in the representation. To validate our approach, we simulated a virtual 6-degree of freedom robot performing a spiral-like survey of a sphere, and provide real-world experimental results for an autonomous underwater vehicle used for automated ship hull inspection. We demonstrate that using the sparse 3D point cloud greatly improves the self-consistency of the map. Furthermore, the use of our piecewise-planar framework provides an additional constraint to multi-session underwater SLAM, improving performance over monocular camera measurements alone.