We present a novel approach to 6DOF pose estimation and segmentation of rigid 3D objects using a single monocular RGB camera based on temporally consistent, local color histograms. We show that this approach outperforms previous methods in cases of cluttered backgrounds, heterogenous objects, and occlusions. The proposed histograms can be used as statistical object descriptors within a template matching strategy for pose recovery after temporary tracking loss e.g. caused by massive occlusion or if the object leaves the camera’s field of view. The descriptors can be trained online within a couple of seconds moving a handheld object in front of a camera. During the training stage, our approach is already capable to recover from accidental tracking loss. We demonstrate the performance of our method in comparison to the state of the art in different challenging experiments including a popular public data set.

Real-Time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms

We present a system for generating and visualizing interactive 3D Augmented Reality tutorials based on 2D video input, which allows viewpoint control at runtime. Inspired by assembly planning, we analyze the input video using a 3D CAD model of the object to determine an assembly graph that encodes blocking relationships between parts. Using an assembly graph enables us to detect assembly steps that are otherwise difficult to extract from the video, and generally improves object detection and tracking by providing prior knowledge about movable parts. To avoid information loss, we combine the 3D animation with relevant parts of the 2D video so that we can show detailed manipulations and tool usage that cannot be easily extracted from the video. To further support user orientation, we visually align the 3D animation with the real-world object by using texture information from the input video. We developed a presentation system that uses commonly available hardware to make our results accessible for home use and demonstrate the effectiveness of our approach by comparing it to traditional video tutorials.

Video-Annotated Augmented Reality Assembly Tutorials

In this paper we propose a robust edge-based approach for 3D textureless object tracking. We first introduce an edge-based pose estimation method, which minimizes the holistic distance between the projected object contour and the query image edges, without explicitly searching for 3D-2D correspondences. This method is accurate with a good initialization; however, it is sensitive to occlusion and fast motion, thus often gets lost in real environments. To improve robustness, we exploit consistency of edge direction for validating the correctness of the estimated 3D pose, and further incorporate the validation scheme for robust estimation, non-local searching and failure recovery. The robust estimation adopts point-wise validation to reduce the effect of outlier, resulting in a direction-based robust estimator. The non-local searching is based on particle filter, with the pose validation for a faithful weighting of particles, which is shown to be better than the distance-based weighting. The failure recovery is based on fast 2D detection, and estimates the recovered pose by searching for 3D-2D point correspondences, with the validation scheme to adaptively determine state transition. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and background clutters.

Robust edge-based 3D object tracking with direction-based pose validation

Due to the lack of distinctiveness to the background, robust monocular 6DOF pose tracking of less-distinct objects remains an important open problem. In this paper, we firstly analyze the object distinctiveness in the tracking process. Then, we propose a novel contour-part model based robust monocular pose tracking method for less-distinct objects. This paper uses the traditional contour feature for pose tracking in a novel strategy called contour part model. First, the contour part model is built by segmenting the projected contour rendered from the 3D model into contour segments of a certain length adaptively according to the Shi-Tomasi cornerness scores. Then, the correspondence is detected in the input image for each contour part by gradient orientation based template matching. Finally, pose tracking is achieved by solving the PnP problem. Experiment results on semi-synthetic and real images show that the proposed method performs better than the existing methods when pose tracking less-distinct objects and shows great robustness toward interference. Additionally, we combine the edge feature and the regional feature in a simple strategy. The overall performance of the fused method is boosted according to the experimental results on the RBOT dataset. The fact shows that the fusion of the edge and regional features has great potential for improving tracking performance.

Robust Monocular Pose Tracking of Less-Distinct Objects Based on Contour-Part Model

Currently, marker-based tracking is the most used method to develop augmented reality (AR) applications (apps). However, this method cannot be applied in some complex and outdoor settings such as prehistoric rock art sites owing to the fact that the usage of markers is restricted on site. Thus, natural feature tracking methods have to be used. There is a wide range of libraries to develop AR apps based on natural feature tracking. In this paper, a comparative study of Vuforia and ARToolKit libraries is carried out, analysing factors such as distance, occlusion and lighting conditions that affect user experience in both indoor and outdoor environments, and eventually the app developer. Our analysis confirms that Vuforia’s user experience indoor is better, faster and flicker-free whether the images are properly enhanced, but it does not work properly on site. Therefore, the development of AR apps for complex outdoor environments such as rock art sites should be performed with ARToolKit.

/pdf/augmented-reality-application-assessment-for-disseminating-11gj91l633.pdf

Augmented reality application assessment for disseminating rock art

Textureless 3D object tracking of the object's position and orientation is a considerably challenging problem, for which a 3D model is commonly used. The 3D---2D correspondence between a known 3D object model and 2D scene edges in an image is standardly used to locate the 3D object, one of the most important problems in model-based 3D object tracking. State-of-the-art methods solve this problem by searching correspondences independently. However, this often fails in highly cluttered backgrounds, owing to the presence of numerous local minima. To overcome this problem, we propose a new method based on global optimization for searching these correspondences. With our search mechanism, a graph model based on an energy function is used to establish the relationship of the candidate correspondences. Then, the optimal correspondences can be efficiently searched with dynamic programming. Qualitative and quantitative experimental results demonstrate that the proposed method performs favorably compared to the state-of-the-art methods in highly cluttered backgrounds.

Global optimal searching for textureless 3D object tracking

This paper presents a monocular model-based 3D tracking approach for textureless objects. Instead of explicitly searching for 3D-2D correspondences as previous methods, which unavoidably generates individual outlier matches, we aim to minimize the holistic distance between the predicted object contour and the query image edges. We propose a method that can directly solve 3D pose parameters in unsegmented edge distance field. We derive the differentials of edge matching distance with respect to the pose parameters, and search the optimal 3D pose parameters using standard gradient-based non-linear optimization techniques. To avoid being trapped in local minima and to deal with potential large inter-frame motions, a particle filtering process with a first order autoregressive state dynamics is exploited. Occlusions are handled by a robust estimator. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and cluttered backgrounds.

Pose optimization in edge distance field for textureless 3D object tracking

In this paper, we introduce an online video-based system that actively assists users in assembly tasks. The system guides and monitors the assembly process by providing instructions and feedback on possibly erroneous operations, enabling easy and effective guidance in AR/MR applications. The core of our system is an online video-based assembly parsing method that can understand the assembly process, which is known to be extremely hard previously. Our method exploits the availability of the participating parts to significantly alleviate the problem, reducing the recognition task to an identification problem, within a constrained search space. To further constrain the search space, and understand the observed assembly activity, we introduce a tree-based global-inference technique. Our key idea is to incorporate part-interaction rules as powerful constraints which significantly regularize the search space and correctly parse the assembly video at interactive rates. Complex examples demonstrate the effectiveness of our method.

Active Assembly Guidance with Online Video Parsing

The invention relates to a monocular texture-free three-dimensional object attitude tracking method based on a three-dimensional geometric model. The method of the invention uses the edge features ofthe object to carry out attitude estimation, does not depend on the feature points of the object surface, and is suitable for weak texture or texture-free three-dimensional object tracking. The methodof the invention can solve the attitude parameters of a texture-free three-dimensional object (the rotation and translation parameters of the object relative to the camera) in real time without installing a mark or a hardware positioning sensor on the three-dimensional object, the virtual object generated by computer is aligned with the three-dimensional object in the image sequence, and the virtual-real superposition effect of seamless fusion is generated.

A monocular texture-free three-dimensional object attitude tracking method based on a three-dimensional geometric model.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Bin Wang

Author Tools

Chat about Author