Integration of information from face and voice plays a central role in our social interactions. It has been mostly studied in the context of audiovisual speech perception: integration of affective or identity information has received comparatively little scientific attention. Here, we review behavioural and neuroimaging studies of face-voice integration in the context of person perception. Clear evidence for interference between facial and vocal information has been observed during affect recognition or identity processing. Integration effects on cerebral activity are apparent both at the level of heteromodal cortical regions of convergence, particularly bilateral posterior superior temporal sulcus (pSTS), and at 'unimodal' levels of sensory processing. Whether the latter reflects feedback mechanisms or direct crosstalk between auditory and visual cortices is as yet unclear.

Integrating Face and Voice in Person Perception

Spoken language exists because of a remarkable neural process. Inside a speaker's brain, an intended message gives rise to neural signals activating the muscles of the vocal tract. The process is remarkable because these muscles are activated in just the right way that the vocal tract produces sounds a listener understands as the intended message. What is the best approach to understanding the neural substrate of this crucial motor control process? One of the key recent modeling developments in neuroscience has been the use of state feedback control (SFC) theory to explain the role of the CNS in motor control. SFC postulates that the CNS controls motor output by (1) estimating the current dynamic state of the thing (e.g., arm) being controlled, and (2) generating controls based on this estimated state. SFC has successfully predicted a great range of non-speech motor phenomena, but as yet has not received attention in the speech motor control community. Here, we review some of the key characteristics of speech motor control and what they say about the role of the CNS in the process. We then discuss prior efforts to model the role of CNS in speech motor control, and argue that these models have inherent limitations - limitations that are overcome by an SFC model of speech motor control which we describe. We conclude by discussing a plausible neural substrate of our model.

https://escholarship.org/content/qt99k491c8/qt99k491c8.pdf?t=qag86d

Speech production as state feedback control.

Large individual differences in spoken word recognition performance have been found in deaf children after cochlear implantation. Recently, Pisoni and Geers (2000) reported that simple forward digit span measures of verbal working memory were significantly correlated with spoken word recognition scores even after potentially confounding variables were statistically controlled for. The present study replicates and extends these initial findings to the full set of 176 participants in the CID cochlear implant study. The pooled data indicate that despite statistical "partialling-out" of differences in chronological age, communication mode, duration of deafness, duration of device use, age at onset of deafness, number of active electrodes, and speech feature discrimination, significant correlations still remain between digit span and several measures of spoken word recognition. Strong correlations were also observed between speaking rate and both forward and backward digit span, a result that is similar to previously reported findings in normal-hearing adults and children. The results suggest that perhaps as much as 20% of the currently unexplained variance in spoken word recognition scores may be independently accounted for by individual differences in cognitive factors related to the speed and efficiency with which phonological and lexical representations of spoken words are maintained in and retrieved from working memory. A smaller percentage, perhaps about 7% of the currently unexplained variance in spoken word recognition scores, may be accounted for in terms of working memory capacity. We discuss how these relationships may arise and their contribution to subsequent speech and language development in prelingually deaf children who use cochlear implants.

/pdf/measures-of-working-memory-span-and-verbal-rehearsal-speed-2hydmbywub.pdf

Measures of working memory span and verbal rehearsal speed in deaf children after cochlear implantation.

This study focused on the long-term speech perception performances of 34 prelingually deafened children who received multichannel cochlear implants manufactured by Cochlear Corporation. The childre...

/pdf/cochlear-implant-use-by-prelingually-deafened-children-the-13zmke1hoi.pdf

Cochlear Implant Use by Prelingually Deafened Children: The Influences of Age at Implant and Length of Device Use

When motor actions (e.g., reaching with your hand) adapt to altered sensory feedback (e.g., viewing a shifted image of your hand through a prism), the phenomenon is called sensorimotor adaptation (...

/pdf/sensorimotor-adaptation-of-speech-i-compensation-and-gm4v45tixu.pdf

Sensorimotor adaptation of speech I: Compensation and adaptation.

A picture identification test for measuring speech discrimination ability in hearing-impaired children was developed in two phases. In the first phase the word stimuli were evaluated to determine whether they were within the recognition vocabulary of the children and whether the pictorial representations of the words were adequate. Before the second phase, the test was revised to consist of 25 plates with 6 pictures on each plate, with only 4 of the pictures on each plate used as test stimuli. These 4 lists were given to 61 hearing-impaired children on two separate occasions. The results indicate reliability coefficients in excess of 0.87 for all four lists, with mean differences of less than 3% and correlation coefficients between lists greater than 0.84. The test appears to be a potentially valuable clinical tool in pediatric audiology. We call it the Word Intelligibility by Picture Identification test (WIPI).

A picture identification test for hearing-impaired children.

/pdf/a-picture-identification-test-for-hearing-impaired-children-17snydqfsy.pdf

A Picture-Identification Test for Hearing-Impaired Children. Final Report.

Word Intelligibility by Picture Identification.

Auditory Management of Hearing Impaired Children

The relationships between each of seven predictor variables and the relative degree to which 84 normal and hearing-impaired children used audition or vision in their perception of word stimuli were investigated. The children's relative use of audition or vision was assessed by the auditory-visual presentation of monosyllabic word stimuli in which the visual word stimuli were in conflict with those presented acoustically. Six of the seven predictor variables were significantly correlated with the performance scores obtained within the auditory-visual conflict condition. Only pure-tone average hearing level and auditory word identification performance, however, made unique contributions toward predicting the degree to which audition or vision was used in the perception of the word stimuli. We concluded that the relative use of audition or vision was almost completely related to their auditory capabilities as represented by the children's unaided threshold sensitivity and aided speech reception performance.

Primary modality for speech perception in children with normal and impaired hearing.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Mark Ross

Author Tools

Chat about Author