Abstract: Accurate localization of a vehicle is a fundamental challenge and one of the most important tasks of mobile robots For autonomous navigation, motion tracking, and obstacle detection and avoidance, a robot must maintain knowledge of its position over time Vision-based odometry is a robust technique utilized for this purpose It allows a vehicle to localize itself robustly by using only a stream of images captured by a camera attached to the vehicle This paper presents a review of state-of-the-art visual odometry (VO) and its types, approaches, applications, and challenges VO is compared with the most common localization sensors and techniques, such as inertial navigation systems, global positioning systems, and laser sensors Several areas for future research are also highlighted

Abstract: We propose a novel direct visual-inertial odometry method for stereo cameras. Camera pose, velocity and IMU biases are simultaneously estimated by minimizing a combined photometric and inertial energy functional. This allows us to exploit the complementary nature of vision and inertial data. At the same time, and in contrast to all existing visual-inertial methods, our approach is fully direct: geometry is estimated in the form of semi-dense depth maps instead of manually designed sparse keypoints. Depth information is obtained both from static stereo - relating the fixed-baseline images of the stereo camera - and temporal stereo - relating images from the same camera, taken at different points in time. We show that our method outperforms not only vision-only or loosely coupled approaches, but also can achieve more accurate results than state-of-the-art keypoint-based methods on different datasets, including rapid motion and significant illumination changes. In addition, our method provides high-fidelity semi-dense, metric reconstructions of the environment, and runs in real-time on a CPU.

Abstract: Loop-closure detection on 3D data is a challenging task that has been commonly approached by adapting image-based solutions. Methods based on local features suffer from ambiguity and from robustness to environment changes while methods based on global features are viewpoint dependent. We propose SegMatch, a reliable loop-closure detection algorithm based on the matching of 3D segments. Segments provide a good compromise between local and global descriptions, incorporating their strengths while reducing their individual drawbacks. SegMatch does not rely on assumptions of "perfect segmentation", or on the existence of "objects" in the environment, which allows for reliable execution on large scale, unstructured environments. We quantitatively demonstrate that SegMatch can achieve accurate localization at a frequency of 1Hz on the largest sequence of the KITTI odometry dataset. We furthermore show how this algorithm can reliably detect and close loops in real-time, during online operation. In addition, the source code for the SegMatch algorithm will be made available after publication.

Abstract: Most approaches to visual odometry estimates the camera motion based on point features, consequently, their performance deteriorates in low-textured scenes where it is difficult to find a reliable set of them. This paper extends a popular semi-direct approach to monocular visual odometry known as SVO [1] to work with line segments, hence obtaining a more robust system capable of dealing with both textured and structured environments. The proposed odometry system allows for the fast tracking of line segments since it eliminates the necessity of continuously extracting and matching features between subsequent frames. The method, of course, has a higher computational burden than the original SVO, but it still runs with frequencies of 60Hz on a personal computer while performing robustly in a wider variety of scenarios.

Abstract: The perception of the dynamic aspects of the environment is a highly relevant precondition for the realization of autonomous robot system acting in the real world. In this paper, we propose a novel method for estimating dense rigid scene flow in 3D LiDAR scans. We formulate the problem as an energy minimization problem, where we assume local geometric constancy and incorporate regularization for smooth motion fields. Analyzing the dynamics at point level helps in inferring the fine-grained details of motion. We show results on multiple sequences of the KITTI odometry dataset, where we seamlessly estimate multiple motions pertaining to different dynamic objects. Furthermore, we test our approach on a dataset with pedestrians to show how our method adapts to a case with non-rigid motion. For comparison we use the ground truth from KITTI and show how our method outperforms different ICP-based methods.

Abstract: Accurate and efficient self-localization is a critical problem for autonomous systems. This paper describes an affordable solution to vehicle self-localization which uses odometry computed from two video cameras and road maps as the sole inputs. The core of the method is a probabilistic model for which an efficient approximate inference algorithm is derived. The inference algorithm is able to utilize distributed computation in order to meet the real-time requirements of autonomous systems in some instances. Because of the probabilistic nature of the model the method is capable of coping with various sources of uncertainty including noise in the visual odometry and inherent ambiguities in the map (e.g., in a Manhattan world). By exploiting freely available, community developed maps and visual odometry measurements, the proposed method is able to localize a vehicle to 4 m on average after 52 seconds of driving on maps which contain more than 2,150 km of drivable roads.

Abstract: Most existing formation control approaches assume that accurate global or local position measurements of the robots are directly available, without giving details about how to obtain these measurements, or only providing Kalman filter-type estimators to get them without considering effects of the estimation on the closed-loop system stability. Hence, developing formation controllers with position estimators that can guarantee overall closed-loop system stability becomes highly desirable. This technical note presents a new formation tracking controller for the nonholonomic mobile robots without using direct position measurements. To deal with the absence of accurate position measurements, feedback information from a perspective camera, the odometry and Attitude and Heading Reference System (AHRS) sensors is used to design an observer to provide online estimates of the relative position of the follower with respect to the leader. Using Lyapunov stability analysis, we show that the combined observer-controller closed-loop system is stable, and both the formation tracking errors and the relative position estimation errors asymptotically converge to zero. The performance of the proposed scheme is illustrated through experimental results.

Abstract: Localization is a key capability for autonomous vehicles especially in urban scenarios. We propose the use of pole-like landmarks as primary features in these environments, as they are distinct, long-term stable and can be detected reliably with a stereo camera system. Furthermore, the resulting map representation is memory efficient, allowing for easy storage and on-line updates. The localization is performed in real-time by a stereo camera system as a main sensor, using vehicle odometry and an off-the-shelf GPS as secondary information sources. Localization is performed by a particle filter approach, coupled with an Kalman filter for robustness and sensor fusion. This leads to a lateral accuracy below 20 cm in various urban test areas. The system has been included in our autonomous test vehicle and successfully demonstrated the full loop from mapping to autonomous driving.

Abstract: The use of cameras as a sensor for odometry estimation is an active research topic that has seen significant growth in recent years. Most methods, however, are only suitable for standard cameras that rely on reasonable lighting. An alternative to overcome low-light conditions is the use of thermal or long-wave infrared imaging. Although visible spectrum and thermal imaging share many characteristics, it is not straightforward to apply standard visual odometry algorithms to thermal imaging data. In this paper, we propose a practical visual odometry system based on a monocular thermal camera. As monocular odometry suffers from an unknown scale factor, the system performs efficient ground plane detection for targeted feature extraction, such that the scale factor can be reliably estimated if the camera height and pitch are known. We also address the problem of periodic nonuniformity correction, which is a necessary characteristic of thermal cameras that freezes the output potentially for over a second and can severely affect motion estimation. In this sense, we automatically determine appropriate times to perform nonuniformity correction based on the current and predicted camera rotations. Experiments illustrate the applicability of the system and compare it with other state estimation approaches.

Abstract: Swarms of low-cost autonomous robots can potentially be used to collectively perform tasks over very large domains and time scales. Novel robots for swarm applications are currently being developed as a result of recent advances in sensing, actuation, processing, power, and manufacturing. These platforms can be used by researchers to conduct experiments with robot collectives and by educators to include robotic hardware in their curricula. However, existing low-cost robots are specialized and can lack desired sensing, navigation, control, and manipulation capabilities. This letter presents a new mobile robot platform, Pheeno, that is affordable, versatile, and suitable for multirobot research, education, and outreach activities. Users can modify Pheeno for their applications by designing custom modules that attach to its core module. We describe the design of the Pheeno core and a three degree-of-freedom gripper module, which enables unprecedented manipulation capabilities for a robot of Pheeno's size and cost. We experimentally demonstrate Pheeno's ability to fuse measurements from its onboard odometry for global position estimation and use its camera for object identification in real time. We also show that groups of two and three Pheenos can act on commands from a central controller and consistently transport a payload in a desired direction.

Abstract: In this paper, we present a stereo visual-inertial odometry algorithm assembled with three separated Kalman filters, i.e., attitude filter, orientation filter, and position filter. Our algorithm carries out the orientation and position estimation with three filters working on different fusion intervals, which can provide more robustness even when the visual odometry estimation fails. In our orientation estimation, we propose an improved indirect Kalman filter, which uses the orientation error space represented by unit quaternion as the state of the filter. The performance of the algorithm is demonstrated through extensive experimental results, including the benchmark KITTI datasets and some challenging datasets captured in a rough terrain campus.

Abstract: The combination of on-board sensors measurements with different statistical characteristics can be employed in robotics for localization and control, especially in GPS-denied environments. In particular, most aerial vehicles are packaged with low cost sensors, important for aerial robotics, such as camera, a gyroscope, and an accelerometer. In this work, we develop a visual inertial odometry system based on the Unscented Kalman Filter (UKF) acting on the Lie group SE(3), such to obtain an unique, singularity-free representation of a rigid body pose. We model this pose with the Lie group SE(3) and model the noise on the corresponding Lie algebra. Moreover, we extend the concepts used in the standard UKF formulation, such as state uncertainty and modeling, to correctly incorporate elements that do not belong to an Euclidean space such as the Lie group members. In this analysis, we use the parallel transport, which requires us to explicitly consider SE(3) as representing rigid bodies though the use of the affine connection. We present experimental results to show the effectiveness of the proposed approach for state estimation of a quadrotor platform.

Abstract: This tutorial chapter aims to teach the main theoretical concepts and explain the use of ROS Navigation Stack. This is a powerful toolbox to path planning and Simultaneous Localization And Mapping (SLAM) but its application is not trivial due to lack of comprehension of the related concepts. This chapter will present the theory inside this stack and explain in an easy way how to perform SLAM in any robot. Step by step guides, example codes explained (line by line) and also real robot testing will be available. We will present the requisites and the how-to’s that will make the readers able to set the odometry, establish reference frames and its transformations, configure perception sensors, tune the navigation controllers and plan the path on their own virtual or real robots.

Abstract: State estimation is the most critical capability for MAV (Micro-Aerial Vehicle) localization, autonomous obstacle avoidance, robust flight control and 3D environmental mapping. There are three main challenges for MAV state estimation: (1) it can deal with aggressive 6 DOF (Degree Of Freedom) motion; (2) it should be robust to intermittent GPS (Global Positioning System) (even GPS-denied) situations; (3) it should work well both for low- and high-altitude flight. In this paper, we present a state estimation technique by fusing long-range stereo visual odometry, GPS, barometric and IMU (Inertial Measurement Unit) measurements. The new estimation system has two main parts, a stochastic cloning EKF (Extended Kalman Filter) estimator that loosely fuses both absolute state measurements (GPS, barometer) and the relative state measurements (IMU, visual odometry), and is derived and discussed in detail. A long-range stereo visual odometry is proposed for high-altitude MAV odometry calculation by using both multi-view stereo triangulation and a multi-view stereo inverse depth filter. The odometry takes the EKF information (IMU integral) for robust camera pose tracking and image feature matching, and the stereo odometry output serves as the relative measurements for the update of the state estimation. Experimental results on a benchmark dataset and our real flight dataset show the effectiveness of the proposed state estimation system, especially for the aggressive, intermittent GPS and high-altitude MAV flight.

Abstract: In this paper we present a fast and precise method to estimate the planar motion of a lidar from consecutive range scans. For every scanned point we formulate the range flow constraint equation in terms of the sensor velocity, and minimize a robust function of the resulting geometric constraints to obtain the motion estimate. Conversely to traditional approaches, this method does not search for correspondences but performs dense scan alignment based on the scan gradients, in the fashion of dense 3D visual odometry. The minimization problem is solved in a coarse-to-fine scheme to cope with large displacements, and a smooth filter based on the covariance of the estimate is employed to handle uncertainty in unconstraint scenarios (e.g. corridors). Simulated and real experiments have been performed to compare our approach with two prominent scan matchers and with wheel odometry. Quantitative and qualitative results demonstrate the superior performance of our approach which, along with its very low computational cost (0.9 milliseconds on a single CPU core), makes it suitable for those robotic applications that require planar odometry. For this purpose, we also provide the code so that the robotics community can benefit from it.

Abstract: We present a direct visual odometry algorithm for a fisheye-stereo camera. Our algorithm performs simultaneous camera motion estimation and semi-dense reconstruction. The pipeline consists of two threads: a tracking thread and a mapping thread. In the tracking thread, we estimate the camera pose via semi-dense direct image alignment. To have a wider field of view (FoV) which is important for robotic perception, we use fisheye images directly without converting them to conventional pinhole images which come with a limited FoV. To address the epipolar curve problem, plane-sweeping stereo is used for stereo matching and depth initialization. Multiple depth hypotheses are tracked for selected pixels to better capture the uncertainty characteristics of stereo matching. Temporal motion stereo is then used to refine the depth and remove false positive depth hypotheses. Our implementation runs at an average of 20 Hz on a low-end PC. We run experiments in outdoor environments to validate our algorithm, and discuss the experimental results. We experimentally show that we are able to estimate 6D poses with low drift, and at the same time, do semi-dense 3D reconstruction with high accuracy. To the best of our knowledge, there is no other existing semi-dense direct visual odometry algorithm for a fisheye-stereo camera.

Abstract: We present a real-time system that enables a highly capable dynamic quadruped robot to maintain an accurate six-degree-of-freedom pose estimate within a 1.0% error of distance traveled over long distances traversed through complex, dynamic outdoor terrain, during day and night, in the presence of camera occlusion and saturation, and occasional large external disturbances, such as slips or falls. The system fuses a stereo-camera sensor, inertial measurement unit, leg odometry, and optional intermittent GPS position updates with an extended Kalman filter to ensure robust, low-latency performance. To maintain a six-degree-of-freedom local positioning accuracy alongside the global positioning knowledge, two reference frames are used; a local reference frame and a global reference frame, with the former benefiting obstacle detection and mapping and the latter for operator-specified and autonomous way-point following. Extensive experimental results obtained from multiple field tests are presented to illustrate the performance and robustness of the system over hours of continuous runs and hundreds of kilometers of distance traveled in a wide variety of terrains and conditions.

Abstract: Maps are an important component of most robotic navigation systems and building maps under uncertainty is often referred to as simultaneous localization and mapping or SLAM. Most SLAM approaches start from scratch and build a map only based on their own observations and odometry information. In this paper, we address the problem of how additional information can be exploited, for example from OpenStreetMap. We extend the standard graph-based SLAM formulation by relating the nodes of the pose-graph with an existing map. As this paper suggests, we can relate the newly built maps with information from publicly available maps with the laser range finder data from the robot and in this way improve the map quality. We implemented and evaluated our approach using real world data taken in urban environments. We illustrate that our extension to graph-based SLAM provides better aligned maps and adds only a marginal computational overhead.

Abstract: While global navigation satellite systems (GNSS) are the state of the art for localization, in general they are unable to operate inside buildings, and there is currently no well-established solution for indoor localization. In this paper we propose a 3D mobile robot pose (2D position and 1D orientation) estimation system for indoor applications. The system is based on the cooperative sensor fusion of radar, ultrasonic and odometry data using an extended Kalman filter (EKF). A prerequisite for the EKF is an occupancy grid map of the scenario as well as the pose of the reference radar node inside the map. Our system can handle even the kidnapped-robot case as the radar provides absolute localization. We conducted a series of measurements in an office building corridor. We determined the typical position root-mean square error (RMSE) to be less than 15 cm.

Abstract: This paper presents a sound-based online localization method for an in-pipe snake robot with an inertial measurement unit (IMU). In-pipe robots, in particular, snake robots need online localization for autonomous inspection and for remote operator supports. The GPS is denied in a pipeline, and conventional odometry-based localization may deteriorate due to slippage and sudden unintended movements. By putting a microphone on the robot and a loudspeaker at the entrance of the pipeline, their distance can be estimated by measuring the time of flight (ToF) of a reference sound emitted from the loudspeaker. Since the sound propagation path in the pipeline is necessary for estimating the robot location, the proposed sound-based online localization method simultaneously estimates the robot location and the pipeline map by combining the distance obtained by the ToF and orientation estimated by the IMU. The experimental results showed that the error of the distance estimation was less than 7% and the accuracy of the pipeline map was more than 68.0%.

Abstract: This paper proposes a method for combining stereo visual odometry, Light Detection And Ranging (LiDAR) odometry and reduced Inertial Measurement Unit (IMU) including two horizontal accelerometers and one vertical gyro. The proposed method starts with stereo visual odometry to estimate six Degree of Freedom (DoF) ego motion to register the point clouds from previous epoch to the current epoch. Then, Generalized Iterative Closest Point (GICP) algorithm refines the motion estimation. Afterwards, forward velocity and Azimuth obtained by visual-LiDAR odometer are integrated with reduced IMU outputs in an Extended Kalman Filter (EKF) to provide final navigation solution. In this paper, datasets from KITTI (Karlsruhe Institute of Technology and Toyota technological Institute) were used to compare stereo visual odometry, integrated stereo visual odometry and reduced IMU, stereo visual-LiDAR odometry and integrated stereo visual-LiDAR odometry and reduced IMU. Integrated stereo visual-LiDAR odometry and reduced IMU outperforms other methods in urban areas with buildings around. Moreover, this method outperforms simulated Reduced Inertial Sensor System (RISS), which uses simulated wheel odometer and reduced IMU. KITTI datasets do not include wheel odometry data. Integrated RTK (Real Time Kinematic) GPS (Global Positioning System) and IMU was replaced by wheel odometer to simulate the response of RISS method. Visual Odometry (VO)-LiDAR is not only more accurate than wheel odometer, but it also provides azimuth aiding to vertical gyro resulting in a more reliable and accurate system. To develop low-cost systems, it would be a good option to use two cameras plus reduced IMU. The cost of such a system will be reduced than using full tactical MEMS (Micro-Electro-Mechanical Sensor) based IMUs because two cameras are cheaper than full tactical MEMS based IMUs. The results indicate that integrated stereo visual-LiDAR odometry and reduced IMU can achieve accuracy at the level of state of art.

Abstract: This chapter examines how certain properties of the world can be exploited in order for a robot or other device to develop a model of its own motion or pose (position and orientation) relative to an external frame of reference. Although this is a critical problem for many autonomous robotic systems, the problem of establishing and maintaining an orientation or position estimate of a mobile agent has a long history in terrestrial navigation.

Abstract: We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible. We leverage recent advances in Bayesian Convolutional Neural Networks to train and implement a sun detection model that infers a three-dimensional sun direction vector from a single RGB image. Crucially, our method also computes a principled uncertainty associated with each prediction, using a Monte Carlo dropout scheme. We incorporate this uncertainty into a sliding window stereo visual odometry pipeline where accurate uncertainty estimates are critical for optimal data fusion. Our Bayesian sun detection model achieves a median error of approximately 12 degrees on the KITTI odometry benchmark training set, and yields improvements of up to 42% in translational ARMSE and 32% in rotational ARMSE compared to standard VO. An open source implementation of our Bayesian CNN sun estimator (Sun-BCNN) using Caffe is available at https://github. com/utiasSTARS/sun-bcnn-vo

Abstract: In this contribution we introduce a framework for precise vehicle localization in dense urban environments which are characterized by high rates of dynamic and semi-static objects. The proposed localization method is specifically designed to handle inconsistencies between map material and sensor measurements. This is achieved by means of a robust map matching procedure based on the Fourier-Mellin transformation (FMT) for global vehicle pose estimation. Accurate and reliable relative localization is obtained from a LiDAR odometry. Consistency checks based on the cumulative sum (CUSUM) test are instrumented for rejection of inconsistent map matching results from the fusion procedure. Our key contributions are: i) Introduction and adaptation of a spectral map matching procedure based on the FMT for urban automated driving, ii) Presentation of a framework for efficient and robust localization in dense urban environments based on a novel LiDAR odometry, map matching, wheel odometry and GPS, iii) Proposal of a procedure for localization integrity monitoring which leads to significantly increased pose estimation accuracy. Evaluation results show the superior performance of the proposed approach compared to another state-of-the-art localization algorithm for a challenging urban dataset. All maps were recorded two years in advance of the evaluation test run. Furthermore, different LiDAR-based sensor setups were used for mapping and localization.

Abstract: We present a method for odometric localization of humanoid robots using standard sensing equipment, i.e., a monocular camera, an inertial measurement unit (IMU), joint encoders and foot pressure sensors. Data from all these sources are integrated using the prediction-correction paradigm of the Extended Kalman Filter. Position and orientation of the torso, defined as the representative body of the robot, are predicted through kinematic computations based on joint encoder readings; an asynchronous mechanism triggered by the pressure sensors is used to update the placement of the support foot. The correction step of the filter uses as measurements the torso orientation, provided by the IMU, and the head pose, reconstructed by a VSLAM algorithm. The proposed method is validated on the humanoid NAO through two sets of experiments: open-loop motions aimed at assessing the accuracy of localization with respect to a ground truth, and closed-loop motions where the humanoid pose estimates are used in real-time as feedback signals for trajectory control.

Abstract: High-precision localization has drawn more and more attention in recent research of intelligent vehicle systems and autonomous robot navigation technology. In most methods, the approaches are only effective in some specific situations. In other words, these methods can only perform well with obvious features, like tall building walls, road curbs, etc . In this paper, a novel framework for precise localization of autonomous vehicle applying to different scenes especially some typical urban environments is proposed. The main procedures of this method include mapping and localization. During mapping process, inertial measurement unit, odometry, and high-precision GPS are fused together with the data sensed by LIDAR, a high-precision map that could provide global position and pose is generated using rolling window. When localizing, live laser data align with the prior-map. A particle filter based point cloud matching method is utilized here. Based on this, a hierarchical localizing method is proposed, which is more accurate and faster than the original matching method. With this method, the sampling guided by proposal density is propagated upward every hierarchy. Besides that, some accelerating algorithms are utilized to make this approach real time. Finally, decimeter-level localization is achieved in different environments, which is proven by some experiments.

Abstract: Disclosed is an intelligent wheel chair control method based on a brain computer interface and an automatic driving technology. The method comprises the following steps: acquiring current pictures by webcams to perform obstacle localization; generating candidate destinations and waypoints for path planning according to the current obstacle information; performing self-localization of the wheel chair; selecting a destination by a user through the brain computer interface (BCI); planning an optimal path according to the current position of the wheel chair as a starting point and the destination selected by the user as an end point in combination with the waypoints; calculating a position error between the current position of the wheel chair and the optimal path as the feedback of a PID path tracking algorithm; and calculating a reference angular velocity and linear velocity by means of the PID path tracking algorithm and transmitting them to a PID motion controller, converting odometry data from encoders into current angular and linear velocities as a feedback of the PID motion controller, and controlling the driving of the wheel chair in real time to the destination. The intelligent wheel chair control method greatly relieves the mental burden of a user, can adapt to changes in the environment, and improves the self-care ability of patients with severe paralysis.