Adaptive user interface design and analysis using emotion recognition through facial expressions and body posture from an RGB-D sensor.
TL;DR: A combination of both automatic and hybrid AUIs result in significantly positive user experience compared to the manual adaptation, and shows that the hybrid adaptation improves usability in terms of productivity and efficiency.
read more
Abstract: This work presents the design and analysis of an Adaptive User Interface (AUI) for a desktop application that uses a novel solution for the recognition of the emotional state of a user through both facial expressions and body posture from an RGB-D sensor. Six basic emotions are recognized through facial expressions in addition to the physiological state, which is recognized through the body posture. The facial expressions and body posture are acquired in real-time from a Kinect sensor. A scoring system is used to improve recognition by minimizing the confusion between the different emotions. The implemented solution achieves an accuracy rate of above 90%. The recognized emotion is then used to derive an Automatic AUI where the user can use speech commands to modify the User Interface (UI) automatically. A comprehensive user study is performed to compare the usability of an Automatic, Manual, and a Hybrid AUI. The AUIs are evaluated in terms of their efficiency, effectiveness, productivity, and error safety. Additionally, a comprehensive analysis is performed to evaluate the results from the viewpoint of different genders and age groups. Results show that the hybrid adaptation improves usability in terms of productivity and efficiency. Finally, a combination of both automatic and hybrid AUIs result in significantly positive user experience compared to the manual adaptation.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
A Hybrid Multimodal Emotion Recognition Framework for UX Evaluation Using Generalized Mixture Functions
Muhammad Asif Razzaq,Jamil Hussain,Jaehun Bang,Cam-Hao Hua,Fahad Ahmed Satti,Ubaid Ur Rehman,Hafiz Syed Muhammad Bilal,Seongjin Kim,Sungyoung Lee +8 more
TL;DR: A hybrid multimodal emotion recognition (H-MMER) framework using multi-view learning approach for unimodal emotions recognition was presented in this paper , which takes into account the importance of difference between multiple modalities and assigns dynamic weights to them by adapting a more efficient combination process with the application of generalized mixture (GM) functions.
Facial Expression Recognition: Impact of Gender on Fairness and Expressions∗
Cristina Manresa-Yee,S. Ramis-Guarinos,Jose Maria Buades Rubio +2 more
- 07 Sep 2022
TL;DR: This work aims to study how gender biased training datasets alter fairness in FER, and the main outcomes show which facial expressions recognition are more impacted by gender bias.
7
Emotion Recognition from Audio-Visual Information based on Convolutional Neural Network
10 May 2023
TL;DR: In this article , an emotion recognition approach using multiple information sources is proposed, which involves both visual and speech information and it is based on Convolutional Neural Networks (CNNs).
4
The Measurement Model User Experience and Technology Acceptance for Implementation Virtual Booth in Marketplace
Henry Antonius Eka Widjaja,Meyliana,Erick Fernando,Stephen W. Santoso,Surjandy,Andreas Raharto Condrobimo +5 more
- 19 Oct 2022
TL;DR: In this article , a measurement model based on TAM and user experience theory is proposed for virtual booths integration in the marketplace, which is tested for feasibility with its validity and reliability with SEM analysis with SMART PLS software.
References
Nasa-Task Load Index (NASA-TLX); 20 Years Later:
Sandra G. Hart
- 01 Oct 2006
TL;DR: The goal was to summarize the environments in which NASA-TLX has been applied, the types of activities the raters performed, other variables that were measured that did (or did not) covary, methodological issues, and lessons learned.
Hand and Mind: What Gestures Reveal About Thought.:
Abstract: The argument of this original and difficult book is that “gestures are an integral part of language as much as are words, phrases and sentences-gestures and language are one system” (p. 2) . Gestures are instantaneous, imagistic, analog, holistic expressions of the same thought that speech renders in hierarchical, linear, digital, analytic form. David McNeill credits Adam Kendon (1972, 1980) with discovering the link between, and essential unity of, speech sounds and gestural movements; his own work elaborates this insight at the higher linguistic levels of semantics and pragmatics. The topic of the book, then, is gestures that accompany speech, the left-hand end of what McNeill calls “ K e n h i ’ s coiitiiiiiiim: Gesticulation + Language-like gestures + Pantomimes 3 Emblems + Sign languages” (p. 37). The continuum ranges from the informal, spontaneous, idiosyncratic movements of the hands and arms that often accompany speech, to the socially-regulated, standardized, linguistic forms of a sign language, with its arbitrary (non-iconic) lexicon. Between these poles the obligatory presence of speech declines and the linguistic properties of gestures increase. “Language-like gestures” are grammatically integrated into an utterance, as when a speaker, asked about the weather on his vacation, replies: “Well, it was [oscillating hand gesture]”, where the “so-so” gesture replaces an adjectival predicate. “Pantomime” conveys its full meaning in silence or, at most, with inarticulate onomatopoeia; also, in pantomime, sequences of gestures can form a unit, as they can in a sign language, but cannot in gesticulation. “Emblems” conform to standards of wellformedness, a language-like property that gesticulation and pantomime lack: in England, the palm-front V-sign is Churchill’s “Victory!”, the palm-back V-sign is a sexual insult. (For an amusing cross-class confusion in emblem dialects, see Collett, Marsh, and O’Shaughnessy, 1979, p. 229, where Margaret Thatcher appears in an Associated Press Photo, making the palm-back V-sign at a moment of electoral triumph.) The contrast between the two ends of Kendon’s continuum, between spontaneous gesture and conventional sign, epitomizes McNeill’s notion of the process by which an utterance evolves in a speaker’s mind. Spontaneous gesture reveals the primitive stage of an utterance, global, unsegmented, non-hierarchical, from which its conventional representation in speech unfolds: hierarchical, segmented, linear. The inner symbols of the primitive stage are private, idiosyncratic, closed to social influence; the end stage is public, grammatical, socially regulated. McNeill supposes that the primitive
2.3K
The computation of optical flow
TL;DR: The computation of optical flow is investigated in this survey: widely known methods for estimating optical flow are classified and examined by scrutinizing the hypothesis and assumptions they use.
A review of affective computing
TL;DR: This first of its kind, comprehensive literature review of the diverse field of affective computing focuses mainly on the use of audio, visual and text information for multimodal affect analysis, and outlines existing methods for fusing information from different modalities.
1.4K
AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität
Marc Hassenzahl,Michael Burmester,Franz Koller +2 more
- 01 Jan 2003
TL;DR: In diesem Beitrag wird der „AttrakDiff 2“ Fragebogen vorgestellt, der sowohl wahrgenommene pragmatische als auch hedonische Qualitat zu messen vermag, ergebnisse zur Reliabilitat and Validitat werden vorgESTellt and diskutiert.
1K