A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
2.5K
TL;DR: In this paper, the authors discuss human emotion perception from a psychological perspective, examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data.
read more
Abstract: Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression
Patrick Lucey,Jeffrey F. Cohn,Takeo Kanade,Jason Saragih,Zara Ambadar,Iain Matthews +5 more
- 13 Jun 2010
TL;DR: The Cohn-Kanade (CK+) database is presented, with baseline results using Active Appearance Models (AAMs) and a linear support vector machine (SVM) classifier using a leave-one-out subject cross-validation for both AU and emotion detection for the posed data.
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Sander Koelstra,Christian Mühl,Mohammad Soleymani,Jong-Seok Lee,Ashkan Yazdani,Touradj Ebrahimi,Thierry Pun,Anton Nijholt,Ioannis Patras +8 more
TL;DR: A multimodal data set for the analysis of human affective states was presented and a novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool.
Multimodal Machine Learning: A Survey and Taxonomy
TL;DR: This paper surveys the recent advances in multimodal machine learning itself and presents them in a common taxonomy to enable researchers to better understand the state of the field and identify directions for future research.
3.4K
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
TL;DR: This survey explicitly explores the multidisciplinary foundation that underlies all AC applications by describing how AC researchers have incorporated psychological theories of emotion and how these theories affect research questions, methods, results, and their interpretations.
OpenFace: An open source facial behavior analysis toolkit
Tadas Baltrusaitis,Peter Robinson,Louis-Philippe Morency +2 more
- 07 Mar 2016
TL;DR: OpenFace is the first open source tool capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation and allows for easy integration with other applications and devices through a lightweight messaging system.
References
Authentic facial expression analysis
Nicu Sebe,Michael S. Lew,Ira Cohen,Yafei Sun,Theo Gevers,Thomas S. Huang +5 more
- 17 May 2004
TL;DR: This paper presents the effort in creating an authentic facial expression database based on spontaneous emotions derived from the environment, and test and compare a wide range of classifiers from the machine learning literature that can be used for facial expression classification.
Machine Analysis of Facial Expressions
Maja Pantic,Bartlett +1 more
- 01 Jul 2007
TL;DR: The human face is a multi-signal input-output communicative system capable of tremendous flexibility and specificity and is the authors' preeminent means of communicating and understanding somebody’s affective state and intentions on the basis of the shown facial expression.
Manifold based analysis of facial expression
TL;DR: Preliminary experimental results show that the probabilistic facial expression model on manifold significantly improves facial deformation tracking and expression recognition.
290
•Proceedings Article
Emotion Recognition in Spontaneous Speech Using GMMs
Daniel Neiberg,Kjell Elenius,Kornel Laskowski +2 more
- 01 Jan 2006
TL;DR: The results indicate that using Gaussian mixture models on the frame level is a feasible technique for emotion classification, and combining the three classifiers significantly improves performance.
A probabilistic framework for modeling and real-time monitoring human fatigue
Qiang Ji,P. Lan,Carl G. Looney +2 more
- 01 Sep 2006
TL;DR: A probabilistic framework based on the Bayesian networks for modeling and real-time inferring human fatigue by integrating information from various sensory data and certain relevant contextual information is introduced, leading to a more robust and accurate fatigue modeling and inference.