Six degree-of-freedom (6-DoF) visual tracking of dynamic objects is fundamental to a large variety of robotics and augmented reality (AR) applications. A key to this problem is accurate distance measurement of dynamic objects, which is usually obtained via stereo cameras, RGB-D sensors, or LiDARs. In this paper, however, we address the problem using only a monocular camera rigidly mounted with a low-cost inertial measurement unit. This is a light-weight, small-size, and low-cost solution, which is particularly suitable for tracking dynamic objects on drones or on mobile phones. Starting from a generic image-based two-dimensional tracker, we propose a novel method to resolve the object scale ambiguity in monocular vision in a geometric manner based on correlation analysis. This enables accurate metric three-dimensional tracking of arbitrary objects without requiring any prior knowledge about the object shape or size. We discuss the applicability by analyzing the observability condition and degenerated cases for object scale recovery. Simulation and real-world experimental results with ground truth comparison, along with AR application examples, demonstrate the feasibility of the proposed 6-DoF tracking method.

Tracking 3-D Motion of Dynamic Objects Using Monocular Visual-Inertial Sensing

A monocular 3D object tracking system generally has only up-to-scale pose estimation results without any prior knowledge of the tracked object. In this paper, we propose a novel idea to recover the metric scale of an arbitrary dynamic object by optimizing the trajectory of the objects in the world frame, without motion assumptions. By introducing an additional constraint in the time domain, our monocular visual-inertial tracking system can obtain continuous six degree of freedom (6-DoF) pose estimation without scale ambiguity. Our method requires neither fixed multi-camera nor depth sensor settings for scale observability, instead, the IMU inside the monocular sensing suite provides scale information for both camera itself and the tracked object. We build the proposed system on top of our monocular visual-inertial system (VINS) to obtain accurate state estimation of the monocular camera in the world frame. The whole system consists of a 2D object tracker, an object region-based visual bundle adjustment (BA), VINS and a correlation analysis-based metric scale estimator. Experimental comparisons with ground truth demonstrate the tracking accuracy of our 3D tracking performance while a mobile augmented reality (AR) demo shows the feasibility of potential applications.

Estimating Metric Poses of Dynamic Objects Using Monocular Visual-Inertial Fusion

This letter studies the problem of target localization in aerial manipulation tasks. When an aerial robot flies close to a target to manipulate, the target would be occluded by the onboard robotic manipulator occasionally or for a long period of time. It is, however, necessary to continuously localize the target even when it is occluded. This is a unique problem in aerial manipulation and has not been addressed in other tasks such as ground manipulation. To solve this problem, we propose a new approach of target-centered range-only mapping based on RGB-D measurements. Different from the traditional map representation where the 3D locations of the feature points are stored in a global reference coordinate frame, we store the distances from the target to the environmental feature points only. It thus results in a coordinate-free target-centered range-only map. When the target is occluded, it can still be localized accurately based on the map. The proposed approach is verified by both synthetic and real datasets. The experimental results demonstrate the superior performance of our proposed approach to the state-of-the-art RGB-D ORB-SLAM3 method.

Robust Localization of Occluded Targets in Aerial Manipulation Via Range-Only Mapping

Artificial neural network (ANN) which has four or more layers structure is called deep NN (DNN) and it is recognized as one of promising machine learning techniques. Convolutional neural network (CNN) is widely used and powerful structure for image recognition and/or defect inspection. It is also known that support vector machine (SVM) has a superior ability for binary classification in spite of only having two layers. The authors already have developed a CNN & SVM design and training tool for defect detection of resin molded articles, while the effectiveness and the validity have been proved through several CNNs design, training and evaluation. The tool further enables to facilitate the design of a CNN model based on transfer learning concept. In this paper, a pick and place robot is introduced while implementing a visual feedback control and a transfer learning-based CNN. The visual feedback control enables to omit the complicated calibration between image and robot coordinate systems, also the transfer learning-based CNN allows the robot to estimate the orientation of target objects for dexterous picking operation. The usefulness and validity of the system is confirmed through pick and place experiments using a small articulated robot named DOBOT.

Pick and Place Robot Using Visual Feedback Control and Transfer Learning-Based CNN

Camera-based stereo-vision provides cost-efficient vision capabilities for robotic systems. The objective of this paper is to examine the performance of stereo-vision as means to enable a robotic inspection cell for haptic quality testing with the ability to detect relevant information related to the inspection task. This information comprises the location and 3D representation of a complex object under inspection as well as the location and type of quality features which are subject to the inspection task. Among the challenges is the low-distinctiveness of features in neighboring area, inconsistent lighting, similar colors as well as low intra-class variances impeding the retrieval of quality characteristics. The paper presents the general outline of the vision chain as well as performance analysis of various algorithms for relevant steps in the machine vision chain thus indicating the capabilities and drawbacks of a camera-based stereo-vision for flexible use in complex machine vision tasks.

Stereo-vision for autonomous industrial inspection robots

Object pose estimation is one of the crucial parts in vision-based object manipulation system using standard industrial robot manipulator, particularly in pose estimation of the end effector of the robot arm to grasp the object targeted. This paper presents the utilization of stereo vision system to estimate the 3D (3 dimensional) object position and orientation to pick up and place the object targeted in an arbitrary location within the workspace. In order to accomplish this task, a calibrated stereo camera in the eye to hand configuration is used to capture the images of the object on the left and right camera. Then, the specific object feature is extracted and the 3D position and orientation of the object are calculated using image processing algorithm. Finally, the end effector of robot arm equipped with gripper will pick up the object targeted according to the object pose estimation output, and then place it to the desired location. The experimental results using 6 DOF robot arm are demonstrated and show the effectiveness of the proposed approach with good performance.

3D object pose estimation using stereo vision for object manipulation system

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Taryudi

Author Tools

Chat about Author

Papers

3D object pose estimation using stereo vision for object manipulation system