Open AccessProceedings Article
Machine Learning for Video-Based Rendering
Arno Schödl,Irfan Essa +1 more
- 01 Jan 2000
- Vol. 13, pp 1002-1008
TL;DR: This work extends the new paradigm for computer animation, video textures, which uses recorded video to generate novel animations by replaying the video samples in a new order by focusing on video sprites, which are a special type of video texture.
read more
Abstract: We present techniques for rendering and animation of realistic scenes by analyzing and training on short video sequences. This work extends the new paradigm for computer animation, video textures, which uses recorded video to generate novel animations by replaying the video samples in a new order. Here we concentrate on video sprites, which are a special type of video texture. In video sprites, instead of storing whole images, the object of interest is separated from the background and the video samples are stored as a sequence of alpha-matted sprites with associated velocity information. They can be rendered anywhere on the screen to create a novel animation of the object. We present methods to create such animations by finding a sequence of sprite samples that is both visually smooth and follows a desired path. To estimate visual smoothness, we train a linear classifier to estimate visual similarity between video samples. If the motion path is known in advance, we use beam search to find a good sample sequence. We can specify the motion interactively by precomputing the sequence cost function using Q-learning.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Near-optimal character animation with continuous control
Adrien Treuille,Yongjoon Lee,Zoran Popović +2 more
- 29 Jul 2007
TL;DR: This work automatically compute near-optimal controllers using a low-dimensional basis representation that produce motion that fluidly responds to several dimensions of user control and environmental constraints in realtime.
Reactive pedestrian path following from examples
Ronald Metoyer,Jessica K. Hodgins +1 more
- 08 May 2003
TL;DR: An approach for generating reactive path following based on the user’s examples of the desired behavior and it is shown that simple direction primitives can be recorded and used to build natural, reactive, path-following behaviors.
115
Reactive pedestrian path following from examples
TL;DR: In this article, the authors explore an approach for generating reactive path following based on the user's examples of the desired behavior, which is combined with reactive control methods to produce natural 2D pedestrian trajectories.
64
What is a Research Proposal
Pam Denicolo,Lucinda Becker +1 more
- 01 Jan 2012
TL;DR: The area of optimising compilation is changed from one that is ad hoc and reactive to hardware evolution to one based on solid foundations that drives architectural change whereby the next generation of hardware is co-designed by compiler technology.
45
Strumming to the Beat: Audio-Conditioned Contrastive Video Textures
01 Jan 2022
TL;DR: In this article , a nonparametric approach for infinite video texture synthesis using a representation learned via contrastive learning is introduced, which can combine semantic and audiovisual cues in order to synthesize videos that synchronize well with an audio signal.
References
•Book
Neural networks for pattern recognition
Christopher M. Bishop
- 01 Jan 1995
TL;DR: This is the first comprehensive treatment of feed-forward neural networks from the perspective of statistical pattern recognition, and is designed as a text, with over 100 exercises, to benefit anyone involved in the fields of neural computation and pattern recognition.
Neural Networks for Pattern Recognition
Suresh Kothari,Heekuck Oh +1 more
TL;DR: The chapter discusses two important directions of research to improve learning algorithms: the dynamic node generation, which is used by the cascade correlation algorithm; and designing learning algorithms where the choice of parameters is not an issue.
14.5K
Reinforcement learning: a survey
TL;DR: Central issues of reinforcement learning are discussed, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state.
•Posted Content
Reinforcement Learning: A Survey
TL;DR: A survey of reinforcement learning from a computer science perspective can be found in this article, where the authors discuss the central issues of RL, including trading off exploration and exploitation, establishing the foundations of RL via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state.
5.9K
Video Rewrite: driving visual speech with audio
Christoph Bregler,Michele Covell,Malcolm Slaney +2 more
- 03 Aug 1997
TL;DR: Video Rewrite is the first facial-animation system to automate all the labeling and assembly tasks required to resync existing footage to a new soundtrack.
826