Abstract: People blinded in infancy have sharper listening skills than those who lost their sight later.

Abstract: The acceptance of cochlear implantation as an effective and safe treatment for deafness has increased steadily over the past quarter century. The earliest devices were the first implanted prostheses found to be successful in compensating partially for lost sensory function by direct electrical stimulation of nerves. Initially, the main intention was to provide limited auditory sensations to people with profound or total sensorineural hearing impairment in both ears. Although the first cochlear implants aimed to provide patients with little more than awareness of environmental sounds and some cues to assist visual speech-reading, the technology has advanced rapidly. Currently, most people with modern cochlear implant systems can understand speech using the device alone, at least in favorable listening conditions. In recent years, an increasing research effort has been directed towards implant users' perception of nonspeech sounds, especially music. This paper reviews that research, discusses the published experimental results in terms of both psychophysical observations and device function, and concludes with some practical suggestions about how perception of music might be enhanced for implant recipients in the future. The most significant findings of past research are: (1) On average, implant users perceive rhythm about as well as listeners with normal hearing; (2) Even with technically sophisticated multiple-channel sound processors, recognition of melodies, especially without rhythmic or verbal cues, is poor, with performance at little better than chance levels for many implant users; (3) Perception of timbre, which is usually evaluated by experimental procedures that require subjects to identify musical instrument sounds, is generally unsatisfactory; (4) Implant users tend to rate the quality of musical sounds as less pleasant than listeners with normal hearing; (5) Auditory training programs that have been devised specifically to provide implant users with structured musical listening experience may improve the subjective acceptability of music that is heard through a prosthesis; (6) Pitch perception might be improved by designing innovative sound processors that use both temporal and spatial patterns of electric stimulation more effectively and precisely to overcome the inherent limitations of signal coding in existing implant systems; (7) For the growing population of implant recipients who have usable acoustic hearing, at least for low-frequency sounds, perception of music is likely to be much better with combined acoustic and electric stimulation than is typical for deaf people who rely solely on the hearing provided by their prostheses.

Abstract: Physiological studies of auditory perception have not yet clearly distinguished sensory from decision processes. In this experiment, human participants identified speech sounds masked by varying levels of noise while blood oxygenation signals in the brain were recorded with functional magnetic resonance imaging (fMRI). Accuracy and response time were used to characterize the behavior of sensory and decision components of this perceptual system. Oxygenation signals in a cortical subregion just anterior and lateral to primary auditory cortex predicted accuracy of sound identification, whereas signals in an inferior frontal region predicted response time. Our findings provide neurophysiological evidence for a functional distinction between sensory and decision mechanisms underlying auditory object identification. The present results also indicate a link between inferior frontal lobe activation and response-selection processes during auditory perception tasks.

Abstract: Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification--in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.

Abstract: The brain integrates information from multiple sensory modalities and, through this process, generates a coherent and apparently seamless percept of the external world. Although multisensory integration typically binds information that is derived from the same event, when multisensory cues are somewhat discordant they can result in illusory percepts such as the "ventriloquism effect." These biases in stimulus localization are generally accompanied by the perceptual unification of the two stimuli. In the current study, we sought to further elucidate the relationship between localization biases, perceptual unification and measures of a participant's uncertainty in target localization (i.e., variability). Participants performed an auditory localization task in which they were also asked to report on whether they perceived the auditory and visual stimuli to be perceptually unified. The auditory and visual stimuli were delivered at a variety of spatial (0 degrees, 5 degrees, 10 degrees, 15 degrees ) and temporal (200, 500, 800 ms) disparities. Localization bias and reports of perceptual unity occurred even with substantial spatial (i.e., 15 degrees ) and temporal (i.e., 800 ms) disparities. Trial-by-trial comparison of these measures revealed a striking correlation: regardless of their disparity, whenever the auditory and visual stimuli were perceived as unified, they were localized at or very near the light. In contrast, when the stimuli were perceived as not unified, auditory localization was often biased away from the visual stimulus. Furthermore, localization variability was significantly less when the stimuli were perceived as unified. Intriguingly, on non-unity trials such variability increased with decreasing disparity. Together, these results suggest strong and potentially mechanistic links between the multiple facets of multisensory integration that contribute to our perceptual Gestalt.

Abstract: People often move in synchrony with auditory rhythms (e.g., music), whereas synchronization of movement with purely visual rhythms is rare. In two experiments, this apparent attraction of movement to auditory rhythms was investigated by requiring participants to tap their index finger in synchrony with an isochronous auditory (tone) or visual (flashing light) target sequence while a distractor sequence was presented in the other modality at one of various phase relationships. The obtained asynchronies and their variability showed that auditory distractors strongly attracted participants' taps, whereas visual distractors had much weaker effects, if any. This asymmetry held regardless of the spatial congruence or relative salience of the stimuli in the two modalities. When different irregular timing patterns were imposed on target and distractor sequences, participants' taps tended to track the timing pattern of auditory distractor sequences when they were approximately in phase with visual target sequences, but not the reverse. These results confirm that rhythmic movement is more strongly attracted to auditory than to visual rhythms. To the extent that this is an innate proclivity, it may have been an important factor in the evolution of music.

Abstract: It is now well-established that there is a causal connection between children's phonological skills and their acquisition of reading and spelling. Here we study low-level auditory processes that may underpin the development of phonological representations in children. Dyslexic and control children were given a battery of phonological tasks, reading and spelling tasks and auditory processing tasks. Potential relations between deficits in dyslexic performance in the auditory processing tasks and phonological awareness were explored. It was found that individual differences in auditory tasks requiring amplitude envelope rise time processing explained significant variance in phonological processing. It is argued that developmentally, amplitude envelope cues may be primary in establishing well-specified phonological representations, as these cues should yield important rhythmic and syllable-level information about speech. Copyright © 2004 John Wiley & Sons, Ltd.

Abstract: This study investigated multisensory interactions in the perception of auditory and visual motion. When auditory and visual apparent motion streams are presented concurrently in opposite directions, participants often fail to discriminate the direction of motion of the auditory stream, whereas perception of the visual stream is unaffected by the direction of auditory motion (Experiment 1). This asymmetry persists even when the perceived quality of apparent motion is equated for the 2 modalities (Experiment 2). Subsequently, it was found that this visual modulation of auditory motion is caused by an illusory reversal in the perceived direction of sounds (Experiment 3). This "dynamic capture" effect occurs over and above ventriloquism among static events (Experiments 4 and 5), and it generalizes to continuous motion displays (Experiment 6). These data are discussed in light of related multisensory phenomena and their support for a "modality appropriateness" interpretation of multisensory integration in motion perception.

Abstract: Functional magnetic resonance imaging (fMRI) was used to investigate the representation of sound categories in human auditory cortex. Experiment 1 investigated the representation of prototypical (good) and nonprototypical (bad) examples of a vowel sound. Listening to prototypical examples of a vowel resulted in less auditory cortical activation than did listening to nonprototypical examples. Experiments 2 and 3 investigated the effects of categorization training and discrimination training with novel nonspeech sounds on auditory cortical representations. The 2 training tasks were shown to have opposite effects on the auditory cortical representation of sounds experienced during training: Discrimination training led to an increase in the amount of activation caused by the training stimuli, whereas categorization training led to decreased activation. These results indicate that the brain efficiently shifts neural resources away from regions of acoustic space where discrimination between sounds is not behaviorally important (e.g., near the center of a sound category) and toward regions where accurate discrimination is needed. The results also provide a straightforward neural account of learned aspects of perceptual distortion near sound categories: Sounds from the center of a category are more difficult to discriminate from each other than sounds near category boundaries because they are represented by fewer cells in the auditory cortical areas.

Abstract: It is important to know what cues the sensory system extracts from natural stimuli and how the brain uses them to form perception. To explore this issue, Smith, Delgutte, and Oxenham [Nature (London) 416, 87–90 (2002)] mixed one sound’s temporal envelope with another sound’s fine temporal structure to produce auditory chimaeras and found that “the perceptual importance of the envelope increases with the number of frequency bands, while that of the fine structure diminishes.” This study addressed two technical issues related to natural cochlear filtering and artificial filter ringing in the chimaerizing algorithm. In addition, this study found that the dichotomy in auditory perception revealed by auditory chimaeras is an epiphenomenon of the classic dichotomy between low- and high-frequency processing. Finally, this study found that the temporal envelope determines sound location as long as the interaural level difference cue is present. The present result reinforces the original hypothesis that the temporal envelope is critical for speech perception whereas temporal fine structure is critical for pitch perception, but does not support the assertion regarding the temporal envelope and fine structure as the acoustic basis for the “what” and “where” mechanisms.

Abstract: This experiment investigated the effect of modality on temporal discrimination in children aged 5 and 8 years and adults using a bisection task with visual and auditory stimuli ranging from 200 to 800 ms. In the first session, participants were required to compare stimulus durations with standard durations presented in the same modality (within-modality session), and in the second session in different modalities (cross-modal session). Psychophysical functions were orderly in all age groups, with the proportion of long responses (judgement that a duration was more similar to the long than to the short standard) increasing with the stimulus duration, although functions were flatter in the 5-year-olds than in the 8-year-olds and adults. Auditory stimuli were judged to be longer than visual stimuli in all age groups. The statistical results and a theoretical model suggested that this modality effect was due to differences in the pacemaker speed of the internal clock. The 5-year-olds also judged visual stimuli as more variable than auditory ones, indicating that their temporal sensitivity was lower in the visual than in the auditory modality.

Abstract: In order to ascertain whether the neural system for auditory working memory exhibits a functional dissociation for spatial and nonspatial information, we used functional magnetic resonance imaging and a single set of auditory stimuli to study working memory for the location and identity of human voices. The subjects performed a delayed recognition task for human voices and voice locations and an auditory sensorimotor control task. Several temporal, parietal, and frontal areas were activated by both memory tasks in comparison with the control task. However, during the delay periods, activation was greater for the location than for the voice identity task in dorsal prefrontal (SFS/PreCG) and parietal regions and, conversely, greater for voices than locations in ventral prefrontal cortex and the anterior portion of the insula. This preferential response to the voice identity task in ventral prefrontal cortex continued during the recognition test period, but the double dissociation was observed only during maintenance, not during encoding or recognition. Together, the present findings suggest that, during auditory working memory, maintenance of spatial and nonspatial information modulates activity preferentially in a dorsal and a ventral auditory pathway, respectively. Furthermore, the magnitude of this dissociation seems to be dependent on the cognitive operations required at different times during task performance.

Abstract: Otoacoustic emissions or OAEs (reflections of cochlear energy produced during the processing of sound) were measured in response to two types of stimuli, rapid clicks and sustained tones, in each ear of neonates. OAEs were larger to tones when elicited in the left ear and to clicks when elicited in the right. This finding is similar to those of enhanced processing of tones in right auditory cortical areas and of rapid stimuli on the left, given strong crossed connections from ear to brain. These findings indicate that processing at the level of the ear may facilitate lateralization of auditory function in the brain.

Abstract: Details of brain areas for acoustical Gestalt perception and the recognition of species-specific vocalizations are not known. Here we show how spectral properties and the recognition of the acoustical Gestalt of wriggling calls of mouse pups based on a temporal property are represented in auditory cortical fields and an association area (dorsal field) of the pups' mothers. We stimulated either with a call model releasing maternal behaviour at a high rate (call recognition) or with two models of low behavioural significance (perception without recognition). Brain activation was quantified using c-Fos immunocytochemistry, counting Fos-positive cells in electrophysiologically mapped auditory cortical fields and the dorsal field. A frequency-specific labelling in two primary auditory fields is related to call perception but not to the discrimination of the biological significance of the call models used. Labelling related to call recognition is present in the second auditory field (AII). A left hemisphere advantage of labelling in the dorsoposterior field seems to reflect an integration of call recognition with maternal responsiveness. The dorsal field is activated only in the left hemisphere. The spatial extent of Fos-positive cells within the auditory cortex and its fields is larger in the left than in the right hemisphere. Our data show that a left hemisphere advantage in processing of a species-specific vocalization up to recognition is present in mice. The differential representation of vocalizations of high vs. low biological significance, as seen only in higher-order and not in primary fields of the auditory cortex, is discussed in the context of perceptual strategies.

Abstract: The phenomenon of attentional capture by a unique yet irrelevant singleton distractor has typically been studied in visual search. In this article, the authors examine whether a similar phenomenon occurs in the auditory domain. Participants searched sequences of sounds for targets defined by frequency, intensity, or duration. The presence of a singleton distractor that was unique on an irrelevant dimension (e.g., a low-frequency singleton in search for a target of high intensity) was associated with search costs in both detection and discrimination tasks. However, if the singleton feature coincided with the target item, search was facilitated. These results establish the phenomenon of auditory attentional capture.

Abstract: Seeing a talker's, face influences auditory speech recognition, hut the visible input essential for this influence has yet to be established. Using a new seamless editing technique, the authors examined effects of restricting visible movement to oral or extraoral areas of a talking thee. In Experiment 1, visual speech identification and visual influences on identifying auditory speech were compared across displays in which the whole face moved, the oral area moved, or the extraoral area moved. Visual speech influences on auditory speech recognition were substantial and unchanging across whole-face and oral-movement displays, However, extraoral movement also influenced identification of visual and audiovisual speech. Experiments 2 and 3 demonstrated that these results are dependent on intact and upright facial contexts. but only with extraoral movement displays.

Abstract: Audiovisual perception and imitation are essential for musical learning and skill acquisition. We compared professional pianists to musically naive controls with fMRI while observing piano playing finger–hand movements and serial finger–thumb opposition movements both with and without synchronous piano sound. Pianists showed stronger activations within a fronto-parieto-temporal network while observing piano playing compared to controls and contrasted to perception of serial finger–thumb opposition movements. Observation of silent piano playing additionally recruited auditory areas in pianists. Perception of piano sounds coupled with serial finger–thumb opposition movements evoked increased activation within the sensorimotor network. This indicates specialization of multimodal auditory–sensorimotor systems within a fronto-parieto-temporal network by professional musical training. Musical ''language,'' which is acquired by observation and imitation, seems to be tightly coupled to this network in accord with an observation–execution system linking visual and auditory perception to motor performance.

Abstract: There is a growing consensus that developmental dyslexia is associated with a phonological-core deficit. One symptom of this phonological deficit is a subtle speech-perception deficit. The auditory...

Abstract: Auditory data display is an interdisciplinary field linking auditory perception research, sound engineering, data mining, and human-computer interaction in order to make semantic contents of data perceptually accessible in the form of (nonverbal) audible sound. For this goal it is important to understand the different ways in which sound can encode meaning. We discuss this issue from the perspectives of language, music, functionality, listening modes, and physics, and point out some limitations of current techniques for auditory data display, in particular when targeting high-dimensional data sets. As a promising, potentially very widely applicable approach, we discuss the method of model-based sonification (MBS) introduced recently by the authors and point out how its natural semantic grounding in the physics of a sound generation process supports the design of sonifications that are accessible even to untrained, everyday listening. We then proceed to show that MBS also facilitates the design of an intuitive, active navigation through "acoustic aspects", somewhat analogous to the use of successive two-dimensional views in three-dimensional visualization. Finally, we illustrate the concept with a first prototype of a "tangible" sonification interface which allows us to "perceptually map" sonification responses into active exploratory hand motions of a user, and give an outlook on some planned extensions.

Abstract: The functional significance of the M50 and M100 auditory evoked fields remains unclear. Here we report auditory evoked field data from three different studies employing wide-band noise stimuli. We find that, for the same stimuli, the strength of the M100, as well as its lateralization, are task-modulated. The M50, in contrast, shows three properties: It is dramatically more pronounced for noise stimuli than for pure tones, does not seem to be task dependent, and, is significantly stronger in the left hemisphere in all task conditions. These contrasting patterns of activation shed light on the properties of the response-generating mechanisms and suggest roles in the process of auditory figure-ground segregation.

Abstract: The role of 2 psychological processes, differentiation and organization, were examined in the perception of musical tonality. Differentiation distinguishes elements from one another and was varied in terms of the distribution of pitch durations within tone sequences. Organization establishes relations between differentiated elements and was varied in terms of either conformity with or deviation from a hierarchical description of tonality. Multiple experiments demonstrated that the perception of tonality depended on a minimal degree of differentiation in the distribution of the duration— but not frequency of occurrence— of pitches and only when pitch distributions were hierarchically organized. Moreover, the mere differentiation of the tonic from nontonic pitches was not sufficient to induce tonal percepts. These results are discussed in relation to tonal strength, musical expressiveness, and principles of auditory pattern processing. A fundamental aspect of the perception of visual scenes or auditory sequences is the apprehension of their inherent structural information (e.g., Garner, 1974; Gibson, 1979; Koffka, 1935; Kubovy & Pomerantz, 1981; Lockhead & Pomerantz, 1991). Regardless of the specific information being processed, the basic psychological questions are the same: What types of structure(s) are contained in the scene or sequence? What psychological processes are used to apprehend such structure? What is the observer’s or listener’s sensitivity to this structure? Are different forms of such structure equally accessible, or are some types more easily apprehended? The goal of much of the research in perception and cognition is to explore these basic issues; in the present article, these questions are examined in the context of perceiving music. Music provides an especially compelling arena for investigating such questions. Along with speech, music represents the most complex form of auditory information with which people interact; thus, it is an ideal candidate for both the discovery of fundamental principles of pattern structure and investigating the parameters of listeners’ sensitivity to such information. Moreover, music psychology is in the fortunate position of having entire fields of study devoted to the specification of information available within the musical stimulus itself. For example, work in both musicology and music theory has, over the course of many years, provided elaborate theoretical descriptions of important structural relations existing within music that may or may not be critical in understanding listeners’ percepts of such passages (see Schmuckler, 1997, or Schmuckler & Boltz, 1994, for discussion of this point).