A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
2.5K
TL;DR: In this paper, the authors discuss human emotion perception from a psychological perspective, examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data.
read more
Abstract: Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions despite the fact that deliberate behaviour differs in visual appearance, audio profile, and timing from spontaneously occurring behaviour. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behaviour have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis including audiovisual fusion, linguistic and paralinguistic fusion, and multi-cue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next we examine available approaches to solving the problem of machine understanding of human affective behavior, and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Interpersonal Coordination of HeadMotion in Distressed Couples
TL;DR: The interpersonal coordination of rigid head motion in intimate couples with a history of interpersonal violence was investigated, with head angular displacement and angular velocity for pitch and yaw were strongly correlated, with frequent shifts in lead-lag relationships.
ROC speak: semi-automated personalized feedback on nonverbal behavior from recorded videos
Michelle Fung,Yina Jin,RuJie Zhao,Mohammed Ehsan Hoque +3 more
- 07 Sep 2015
TL;DR: A framework that couples computer algorithms with human intelligence in order to automatically sense and interpret nonverbal behavior and synthesize Mechanical Turk workers' interpretations, ratings, and comment rankings with the machine-sensed data is presented.
44
Multi-label, multi-task CNN approach for context-based emotion recognition
TL;DR: In this article, a new deep learning architecture for context-based multi-label multi-task emotion recognition was proposed, which is built from three main modules: (1) a body features extraction module, which was a pre-trained Xception network, (2) a scene features extract module, based on a modified VGG16 network, and (3) a fusion-decision module.
43
Emotion recognition on speech signals using machine learning
Mohan Ghai,Shamit Lal,Shivam Duggal,Shrey Manik +3 more
- 01 Mar 2017
TL;DR: The prime objective of this paper is to recognize emotions in speech and classify them in 7 emotion output classes namely, anger, boredom, disgust, anxiety, happiness, sadness and neutral.
43
Predicting facial indicators of confusion with hidden Markov models
Joseph F. Grafsgaard,Kristy Elizabeth Boyer,James C. Lester +2 more
- 09 Oct 2011
TL;DR: A predictive model of learner confusion during task-oriented human-human tutorial dialogue that leverages textual dialogue, task, and facial expression history to predict upcoming confusion within a hidden Markov modeling framework is created.
References
Development and validation of brief measures of positive and negative affect: The PANAS scales.
TL;DR: Two 10-item mood scales that comprise the Positive and Negative Affect Schedule (PANAS) are developed and are shown to be highly internally consistent, largely uncorrelated, and stable at appropriate levels over a 2-month time period.
•Book
Affective Computing
Rosalind W. Picard
- 01 Jan 1997
TL;DR: Key issues in affective computing, " computing that relates to, arises from, or influences emotions", are presented and new applications are presented for computer-assisted learning, perceptual information retrieval, arts and entertainment, and human health and interaction.
5.7K
Combining Pattern Classifiers
Ludmila I. Kuncheva
- 02 Jul 2004
TL;DR: This combining pattern classifiers methods and algorithms helps people to enjoy a good book with a cup of coffee in the afternoon, instead they cope with some harmful virus inside their computer.
3K
Comprehensive database for facial expression analysis
Takeo Kanade,Jeffrey F. Cohn,Yingli Tian +2 more
- 26 Mar 2000
TL;DR: The problem space for facial expression analysis is described, which includes level of description, transitions among expressions, eliciting conditions, reliability and validity of training and test data, individual differences in subjects, head orientation and scene complexity image characteristics, and relation to non-verbal behavior.