How many times can a person distinguish between visemes?

Subjective evaluation shows that learners can distinguish either the substitution or deletion of articulation in 79.8% of the time (excluding those chosen “I don’t know”).

What is the blending technique used in the visual-speech synthesizer?

This process is based on the weighted morphing technique [11].(2)where V(t) is the morphing result at time t,w(t) is the blending weight at time t andd is the duration from viseme Va to viseme VbViseme morphing is realized by changing the blending weight from zero to one.

What is the purpose of the test?

1) Part I: Testing for the English phonemes knowledge Part The authoraims to test (i) a subject’s background knowledge of articulation based on the still image, whether they can tell what is being pronounced, (ii) and, whether they can discriminate between possible confusions with the articulation of other phonemes.

Who is the author of the book?

A. M. Harrison, W. Y. Lau, H. Meng and L. Wang, “Improving Mispronunciation Detection and Diagnosis of Learners' Speech with Context-sensitive Phonological Rules based on Language Transfer,” in the Proceedings of Interspeech, Brisbane, Australia, 22-26 September 2008.

How long does it take to close the lips?

For the example /m/ (mat), the authors close the lips very quickly at the beginning and then keep them closed until near the end of the phoneme.

Open AccessProceedings Article10.1109/ISCSLP.2010.5684832

Development of an articulatory visual-speech synthesizer to support language learning

Ka-Ho Wong, +3 more

- 01 Nov 2010

- pp 139-143

TL;DR: An articulatory visual speech synthesizer that can accept free-text input and synthesize articulatory dynamics in real-time is developed that can identify the appropriate word(s) based on articulation animation alone nearly ∼80% of the time.

Abstract: This paper presents a two-dimensional (2D) visual-speech synthesizer to support language learning. A visual-speech synthesizer animates the human articulators in synchronization with speech signals, e.g., output from a text-to-speech synthesizer. A visual-speech animation can offer a concrete illustration to the language learners on how to move and where to place the articulators when pronouncing a phoneme. We adopt a 2D vector-based viseme models and compiled a collection of visemes to cover the articulation of all English phonemes (42 visemes for the 44 English phonemes). Morphing between properly selected vector-based articulation images achieves articulatory animations. In this way, we have developed an articulatory visual speech synthesizer that can accept free-text input and synthesize articulatory dynamics in real-time. Evaluation involving 32 subjects based on “lip-reading” shows that they can identify the appropriate word(s) based on articulation animation alone nearly ∼80% of the time

AI Agents for this Paper

Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps

Most frequently asked questions

1. What are the contributions mentioned in the paper "Development of an articulatory visual-speech synthesizer to support language learning" ?

This paper presents a two-dimensional ( 2D ) visualspeech synthesizer to support language learning.. In this way, the authors have developed an articulatory visual speech synthesizer that can accept free-text input and synthesize articulatory dynamics in real-time.

2. How do the authors control the blending process in the visual-speech synthesizer?

To achieve more realistic and smooth animation, the authors reserve a short period of transition time empirically (e.g., 20% of the phoneme duration) for articulator transitions both at the beginning and at the end of a phoneme.

3. What is the main purpose of the study?

As forprompts with a high percentage of correctness and a low percentage of subjects choosing “I don’t know”, visual-speech may be a useful illustration to instruct the learners of proper articulations.

4. What is the way to test the articulation of the visemes?

The authors adopted vectorbased mid-sagittal representation for the visemes which provides greater flexibility for modification to exaggerate the articulation when needed.

Figure 1: The flow chart illustrated the work of articulatory visual-speech synthesizer.

Figure 4: The two visemes for the diphthong /aʊ/ (e.g., bough). The left viseme shows an open mouth articulation for the /ɑ/ (car) in /aʊ/. The right viseme shows the tongue in a mid, back position with some lip-rounding (/ʊ/ (book) in /aʊ/).

Figure 3: The line extending from the right-hand side of a vertex is right direction handle (1a). The line extending from the left-hand side of a vertex is left direction handle (1b). When we move the handle (2), the shape of curve between two vertices is changed (3).

Figure 2: This is the viseme for /g/ (e.g., in /gap/). The viseme is a vectorized image and the vertices are shown as dots here. We have intentionally assigned a higher density of vertices for the tongue for its flexibility in shape and dynamic movements during articulation.

Table 2: Phoneme represented one or two visemes. The two phonemes (/i/, /u/) are slightly diphthongized.

Figure 5: The sequence of visemes for the phoneme /b/ (as in big). In the animation of /b/, from viseme A transits to C via B1. In this path, we will see the mouth close before the lips are deformed, which is more realistic. However, if the animation changes directly from viseme A to C (i.e. with only the single viseme C being used), the actions at mouth closure and lip deformation will happen simultaneously and will result in an abnormal transition such as B2, which shows an intermediate transition where the lips are deformed before touching.

Citations

•Proceedings Article•10.1109/ICASSP.2011.5947656

Allophonic variations in visual speech synthesis for corrective feedback in CAPT

Ka-Ho Wong, +2 more

- 22 May 2011

TL;DR: A visual speech synthesizer providing midsagittal and front views of the vocal tract to help language learners to correct their mispronunciations is presented.

...read moreread less

•Proceedings Article•10.1109/ICASSP.2013.6639267

Audiovisual synthesis of exaggerated speech for corrective feedback in computer-assisted pronunciation training

Junhong Zhao, +5 more

- 26 May 2013

TL;DR: A novel method for corrective feedback generation, namely, exaggerated feedback, for language learning, which can effectively generate bimodal exaggeration effect for feedback provision and make them more distinctive to be perceived.

...read moreread less

•Journal Article•10.1109/ACCESS.2016.2641474

Smartphone-Assisted Pronunciation Learning Technique for Ambient Intelligence

Jaesung Lee, +3 more

- 01 Jan 2017

- IEEE Access

TL;DR: A smartphone-assisted pronunciation learning technique based on a lightweight word recommendation method that exploits built-in functions supported by IoT members and a computationally moderate word selection method that demonstrates that the user pronunciation is significantly improved without incurring unacceptable computational costs for a smartphone platform.

...read moreread less

•Proceedings Article•10.1109/COGINFOCOM.2013.6719170

Development of text-to-audiovisual speech synthesis to support interactive language learning on a mobile device

Wai-Kim Leung, +3 more

- 01 Dec 2013

TL;DR: Evaluation shows that this newly proposed approach, vis-a-vis the original approach that involves generation of an Ogg video on the server-side which is streamed to the client, achieves a significant reduction in average size of the output files that are transmitted from the server to the Client, reduction of client waiting times, as well as preserve the quality of the image.

...read moreread less

Proceedings Article•10.1109/APSIPA.2013.6694343

Visualization of Mandarin articulation by using a physiological articulatory model

Dian Huang, +6 more

- 01 Oct 2013

TL;DR: This paper proposes a visualization method for Mandarin phoneme pronunciation using a three-dimensional (3D) articulatory physiological model driven by Chinese Electromagnetic Articulographic (EMA) data.

...read moreread less

References

•Book

A course in phonetics

Peter Ladefoged

- 01 Jan 1975

TL;DR: In this paper, the authors introduce articulatory phonetics phonology and phonetic transcription, including the Consonants of English English vowels and English words and sentences, as well as the international phonetic alphabet feature hierarchy performance exercises.

...read moreread less

2.9K

Journal Article•10.1080/13488678.2006.10801179

Asian Englishes: Beyond the Canon

Stephen F. Soresi

- 01 Jun 2006

- Asian Englishes

TL;DR: Kachru as mentioned in this paper argues that English is too often conceptualized as a language in Asia, instead of a language of Asia (9), and proposes to consider the reincarnated English in the Philippines, Singapore and India as a part of our local pluralistic linguistic heritage.

...read moreread less

307

•Journal Article•10.1016/J.SPECOM.2009.05.006

Embodied conversational agents in computer assisted language learning

Preben Wik, +1 more

- 01 Oct 2009

- Speech Communication

TL;DR: This paper describes two systems using embodied conversational agents (ECAs) for language learning, one of which is a virtual language teacher for vocabulary and pronunciation training and the other is a role-playing game for practicing conversational skills.

...read moreread less

150

Proceedings Article•10.1109/ASRU.2007.4430152

Deriving salient learners’ mispronunciations from cross-language phonological comparisons

Helen Meng, +3 more

- 01 Dec 2007

TL;DR: An automatic speech recognizer is developed by training cross-word triphone models based on the TIMIT corpus and an "extended" pronunciation lexicon is developed that incorporates the predicted phonetic confusions to generate additional, erroneous pronunciation variants for each word.

...read moreread less

Journal Article•10.2307/3586075

Pronunciation Contrasts in English

Thomas Scovel, +2 more

- 01 Jun 1972

- TESOL Quarterly

Development of an articulatory visual-speech synthesizer to support language learning

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions mentioned in the paper "Development of an articulatory visual-speech synthesizer to support language learning" ?

2. How do the authors control the blending process in the visual-speech synthesizer?

3. What is the main purpose of the study?

4. What is the way to test the articulation of the visemes?

5. How many times can a person distinguish between visemes?

6. What is the blending technique used in the visual-speech synthesizer?

7. What is the purpose of the test?

8. Who is the author of the book?

9. How long does it take to close the lips?

Figures

Citations

Allophonic variations in visual speech synthesis for corrective feedback in CAPT

Audiovisual synthesis of exaggerated speech for corrective feedback in computer-assisted pronunciation training

Smartphone-Assisted Pronunciation Learning Technique for Ambient Intelligence

Development of text-to-audiovisual speech synthesis to support interactive language learning on a mobile device

Visualization of Mandarin articulation by using a physiological articulatory model

References

A course in phonetics

Asian Englishes: Beyond the Canon

Embodied conversational agents in computer assisted language learning

Deriving salient learners’ mispronunciations from cross-language phonological comparisons

Pronunciation Contrasts in English

Related Papers (5)

Control of an articulatory speech synthesizer based on dynamic approximation of spatial articulatory targets.

Articulatory Speech Re-synthesis: Profiting from Natural Acoustic Speech Data

Articulatory Phonetic Features for Robust Speech Recognition

Control concepts for articulatory speech synthesis

Speech style transfer