Open AccessPosted Content
Learnable Triangulation of Human Pose
TL;DR: Two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views are presented and end-to-end differentiable, which allows us to directly optimize the target metric.
read more
Abstract: We present two novel solutions for multi-view 3D human pose estimation based on new learnable triangulation methods that combine 3D information from multiple 2D views. The first (baseline) solution is a basic differentiable algebraic triangulation with an addition of confidence weights estimated from the input images. The second solution is based on a novel method of volumetric aggregation from intermediate 2D backbone feature maps. The aggregated volume is then refined via 3D convolutions that produce final 3D joint heatmaps and allow modelling a human pose prior. Crucially, both approaches are end-to-end differentiable, which allows us to directly optimize the target metric. We demonstrate transferability of the solutions across datasets and considerably improve the multi-view state of the art on the Human3.6M dataset. Video demonstration, annotations and additional materials will be posted on our project page (this https URL).
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A novel dataset and deep learning-based approach for marker-less motion capture during gait
Saman Vafadar,Wafa Skalli,Aurore Bonnet-Lebrun,Marc Khalifé,Mathis Renaudin,Amine Hamza,Laurent Gajny +6 more
TL;DR: In this article, a marker-less system, based on deep learning-based pose estimation methods, was proposed for gait study, which can contribute to the advancement and evaluation of markerless motion capture systems.
21
LiftPose3D, a deep learning-based approach for transforming 2D to 3D pose in laboratory animals
Adam Gosztolai,Semih Günel,Marco Pietro Abrate,Daniel Morales,Victor Lobato Ríos,Helge Rhodin,Pascal Fua,Pavan Ramdya +7 more
TL;DR: LiftPose3D permits high-quality 3D pose estimation in the absence of complex camera arrays, tedious calibration procedures, and despite occluded keypoints in freely behaving animals.
•Posted Content
Multi-view Human Pose and Shape Estimation Using Learnable Volumetric Aggregation.
Soyong Shin,Eni Halilaj +1 more
TL;DR: This paper proposes a learnable volumetric aggregation approach to reconstruct 3D human body pose and shape from calibrated multi-view images using a parametric representation of the human body, which makes the approach directly applicable to medical applications.
19
HybridTrak: Adding Full-Body Tracking to VR Using an Off-the-Shelf Webcam
Jackie (Junrui) Yang,Tuochao Chen,Fang Qin,Monica S. Lam,James A. Landay +4 more
- 29 Apr 2022
TL;DR: HybridTrak provides accurate, real-time full-body tracking by augmenting inside-out upper-body VR tracking systems with a single external off-the-shelf RGB web camera and is more accurate than RGB or depth-based tracking methods on the MPI-INF-3DHP dataset.
•Posted Content
On the Robustness of Human Pose Estimation
TL;DR: It is found that compared to classification and semantic segmentation, human pose estimation architectures are relatively robust to adversarial attacks with the single-step attacks being surprisingly ineffective and some body-joints are easier to fool than the others.
References
Microsoft COCO: Common Objects in Context
Tsung-Yi Lin,Michael Maire,Serge Belongie,James Hays,Pietro Perona,Deva Ramanan,Piotr Dollár,C. Lawrence Zitnick +7 more
- 06 Sep 2014
TL;DR: A new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding by gathering images of complex everyday scenes containing common objects in their natural context.
•Book
Multiple view geometry in computer vision
Richard Hartley,Andrew Zisserman +1 more
- 01 Jan 2000
TL;DR: In this article, the authors provide comprehensive background material and explain how to apply the methods and implement the algorithms directly in a unified framework, including geometric principles and how to represent objects algebraically so they can be computed and applied.
20.1K
Multiple View Geometry in Computer Vision.
Bernhard P. Wrobel
- 01 Jan 2001
TL;DR: This book is referred to read because it is an inspiring book to give you more chance to get experiences and also thoughts and it will show the best book collections and completed collections.
14.2K
•Proceedings Article
Spatial transformer networks
Max Jaderberg,Karen Simonyan,Andrew Zisserman,Koray Kavukcuoglu +3 more
- 07 Dec 2015
TL;DR: This work introduces a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network, and can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps.
Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
TL;DR: A new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, is introduced for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms.