Multimodal User Interfaces

Question

1. What are the contributions in "Gestural interfaces for hearing-impaired communication" ?

2. What is the way to solve multiclassification problems?

3. What are the common descriptors used for hand shape analysis?

4. What is the effect of the motion in which the hand is hidden?

Accepted Answer

This chapter overviews ( 1 ) the various modalities involved in gestured languages ( 2 ) the mean to automatically apprehend them individually and ( 3 ) to fuse them in order to provide a communication medium adapted to hearing-impaired.. The authors present two example applications, a sign language tutoring tool and a cued speech interpreter and discuss theoretical and practical aspects.

Accepted Answer

An efficient method to solve multiclassification problems is to use a bank of binary classifiers, such as SVM) and to fuse their partial results into a single decision.

Accepted Answer

Region based descriptors (image moments, image eigenvectors, Zernike moments, Hu invariants, or grid descriptors) and edge based descriptors (contour representations, Fourier descriptors, or Curvature Scale Space descriptors) can be used for this purpose.

Accepted Answer

In case the motion in which the static gesture is hidden is a global translation motion (i.e. the change of location in CS or any deixis gesture), any study of the rigid movement is likely to stress the variations of speed and the images on which the motion is small enough to be potentially considered as a target gesture.

Accepted Answer

In order to make sure that the BF associated to each SVM are combinable via the Dempster's rule, it is necessary to apply a refinement from the frame of each SVM (made of two classes), to the frame of the entire set of classes, but then, it provides more accurate results than classical methods.

Accepted Answer

Many of the methods proposed for solving the co-articulation affect, rely on modeling the co-articulation by using pairs or triples of signs instead of a single one.

Accepted Answer

The authors consider that FCS is inherently static: target images are sufficient to decode the continuous sentence: as a consequence, complete transition analysis is most of the time useless to be processed (Burger 2007; Burger et al. 2007a).

Accepted Answer

The centered scale invariant inertial moments of order p+q are calcu-lated as follows:( ) ( ) 12 00with ( , )δ+ += = − −∫∫ p qpq pq pqp qx ym n m x x y y x y dx dymwhere andx y are the coordinates of the center of gravity of the shape and( , ) 1x yδ = if the pixel belongs to the hand shape and 0 otherwise.

Accepted Answer

Among different kinds of HMM architectures, left-to-right HMMs (Fig. 17) with either discrete or continuous observations are preferred for their simplicity and suitability to the hand gesture and sign language recognition problems.

Accepted Answer

In (Burger et al. 2007b), the authors consider the use of a thumb presence indicator, which returns a non-zero positive value if the thumb is unfolded and 0 otherwise.

Accepted Answer

As a consequence, the palm of the hand can be approximated by a circle whose radius is related to the maximum value given by the distance transform of the binary hand image.

Accepted Answer

In case of non-rigid motion, such as the deformation of the hand contour when its shape is considered, it is more difficult to define a cinematic clue that indicates when a target is reached or when the image represents a transitive motion.

Multimodal User Interfaces

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions in "Gestural interfaces for hearing-impaired communication" ?

2. What is the way to solve multiclassification problems?

3. What are the common descriptors used for hand shape analysis?

4. What is the effect of the motion in which the hand is hidden?

5. How can the authors make sure that the BF associated to each SVM is combinable?

6. What is the method for solving the co-articulation affect?

7. is it possible to decode the continuous sentence?

8. What is the centered scale invariant moment of the hand shape?

9. What are the advantages of left-to-right HMMs?

10. What is the common example of a thumb presence indicator?

11. How can the palm of the hand be approximated?

12. What is the difficult way to define a cinematic clue?

Figures

Citations

Brain training with non-action video games enhances aspects of cognition in older adults: a randomized controlled trial

On-line Emotion Recognition in a 3-D Activation-Valence-Time Continuum using Acoustic and Linguistic Cues

The MAHNOB Laughter database

Intelligent in-vehicle interaction technologies

A Real Time Vision-Based Hand Gesture Interaction

References

Snakes : Active Contour Models

Robust Real-Time Face Detection

Visual pattern recognition by moment invariants

The transferable belief model

Automatic facial expression analysis: a survey

Related Papers (5)

Prototyping Multimodal Interfaces with the SMUIML Modeling Language

Multimodal user interfaces : from signals to interaction

Multimodal Interfaces for Augmented Reality

A development environment for the design of multimodal, colourgraphic human-computer interfaces

Multimodal interfaces for automotive applications (MIAA)