Cage-based Performance Capture

TL;DR: This course addresses the hard problem of extracting or acquiring and then reusing non-rigid parametrization for video-based animations in four steps and shows that cage-based deformers offer a flexible design space abstraction to dynamically non- Rigid surface motion through learning space-time shape variability.

Abstract: Nowadays, highly-detailed animations of live-actor performances are increasingly easier to acquire, and 3D Video has reached considerable attention in visual media productions. This lecture will address new paradigm to achieve performance capture using cage-based shapes in motion. We define cage-based performance capture as the non-invasive process of capturing non-rigid surface of actors from multi-view in the form of sparse control deformation handles trajectories and a laser-scanned static template shape.In this course, we address the hard problem of extracting or acquiring and then reusing non-rigid parametrization for video-based animations in four steps: (1) cage-based inverse kinematics, (2) conversion of surface performance capture into cage-based deformation, (3) cage-based cartoon surface exaggeration, and (4) cage-based registration of time-varying reconstructed point clouds. The key objective is to attract the interest of game programmers, digital artists and filmmakers in employing purely geometric and animator-friendly tools to capture and reuse surfaces in motion. Finally, a variety of advanced animation techniques and vision-based graphics applications could benefit from animatable coordinates-based sub-spaces as presented in this course.At first sight, a crucial challenge is to reproduce plausible boneless deformations while preserving global and local captured properties of dynamic surfaces with a limited number of controllable, flexible and reusable parameters. While abandoning the classical articulated skeleton as the underlying structure, we show that cage-based deformers offer a flexible design space abstraction to dynamically non-rigid surface motion through learning space-time shape variability. Registered cage-handles trajectories allow the reconstruction of complex mesh sequences by deforming an enclosed fine-detail mesh. Finally, cage-based performance capture techniques offer suitable and reusable outputs for animation transfer by decoupling the motion from the geometry.