Open Access
Multimodal Adaptive Interfaces
Deb Roy,Alex Pentland +1 more
- 01 Jan 1998
TL;DR: Depending on the task at hand and the user’s preferences, she will use a combination of speech and gesture in different ways to communicate her intent.
read more
Abstract: Speech is the primary mode of communication between people and should also be used in computer human communication. Gesture usually accompanie s speech and provides information which is at times complementary and at times redundant to the information in the speech stream. Depending on the task at hand and the user’s preferences, she will use a combination of speech and gesture in different ways to communicate her intent.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
An Approach to Multi-modal Human-Machine Interaction for Intelligent Service Robots
Hans-Joachim Böhme,Torsten Wilhelm,Jürgen Key,Carsten Schauer,Christof Schröter,Horst-Michael Groß,Torsten Hempel +6 more
TL;DR: A multi-modal scheme for human–robot interaction suited for a wide range of intelligent service robot applications and some reliable methods for vision-based interaction, sound analysis and speech output are developed.
84
A social informatics approach to human-robot interaction with a service social robot
Christine L. Lisetti,Sarah M. Brown,K. Alvarez,Andreas H. Marpaung +3 more
- 01 May 2004
TL;DR: Results have indicated that individuals believe that service robots with emotion and personality capabilities would make them more acceptable in everyday roles in human life, prefer that robots communicate via both human-like facial expressions, voice, and text-based media, and become more positive about the idea of service and social robots after exposure to the technology.
68
Patent
Automatic pronunciation scoring for language learning
Sunil K. Gupta,Ziyi Lu,Fengguang Zhao +2 more
- 03 Jul 2002
TL;DR: This paper proposed a method and apparatus for generating a pronunciation score by receiving a user phrase intended to conform to a reference phrase and processing the user phrase in accordance with at least one of an articulation-scoring engine, a duration scoring engine and an intonation-score engine to derive thereby the pronunciation score.
38
•Proceedings Article
Learning words from natural audio-visual input.
Deb Roy,Alex Pentland +1 more
- 01 Jan 1998
TL;DR: A model of early word learning which learns from natural audio and visual input is presented, which has been successfully implemented to learn words and their audio-visual grounding from camera and microphone input.
27
•Dissertation
Understanding expressive action
Christopher R. Wren,Alex Pentland +1 more
- 01 Jan 2000
TL;DR: This dissertation examines a body of sophisticated perceptual mechanisms developed in response to these needs as well as a selection of human-computer interface sketches designed to push the technology forward and explore the possibilities of this novel interface idiom.
References
•Book
Elements of information theory
Thomas M. Cover,Joy A. Thomas +1 more
- 01 Jan 1991
TL;DR: The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
A tutorial on hidden Markov models and selected applications in speech recognition
Lawrence R. Rabiner
- 01 Feb 1989
TL;DR: In this paper, the authors provide an overview of the basic theory of hidden Markov models (HMMs) as originated by L.E. Baum and T. Petrie (1966) and give practical details on methods of implementation of the theory along with a description of selected applications of HMMs to distinct problems in speech recognition.
Backpropagation through time: what it does and how to do it
Paul J. Werbos
- 01 Jan 1990
TL;DR: This paper first reviews basic backpropagation, a simple method which is now being widely used in areas like pattern recognition and fault diagnosis, and describes further extensions of this method, to deal with systems other than neural networks, systems involving simultaneous equations or true recurrent networks, and other practical issues which arise with this method.
RASTA processing of speech
Hynek Hermansky,Nelson Morgan +1 more
TL;DR: The theoretical and experimental foundations of the RASTA method are reviewed, the relationship with human auditory perception is discussed, the original method is extended to combinations of additive noise and convolutional noise, and an application is shown to speech enhancement.
2.1K
The vocabulary problem in human-system communication
TL;DR: It is shown how this fundamental property of language limits the success of various design methodologies for vocabulary-driven interaction, and an optimal strategy, unlimited aliasing, is derived and shown to be capable of several-fold improvements.
1.6K
Related Papers (5)
Edward Tse,Chia Shen,Saul Greenberg,Clifton Forlines +3 more
- 29 Apr 2007