TL;DR: A system that allows to "touch", grab and manipulate sounds in mid-air, and it is shown that sound localization is surprisingly accurate (11.5 cm), even in the presence of distractors.
Abstract: In this paper we present a system that allows to "touch", grab and manipulate sounds in mid-air. Further, arbitrary objects can seem to emit sound. We use spatial sound reproduction for sound rendering and computer vision for tracking. Using our approach, sounds can be heard from anywhere in the room and always appear to originate from the same (possibly moving) position, regardless of the listener's position. We demonstrate that direct "touch" interaction with sound is an interesting alternative to indirect interaction mediated through controllers or visual interfaces. We show that sound localization is surprisingly accurate (11.5 cm), even in the presence of distractors. We propose to leverage the ventriloquist effect to further increase localization accuracy. Finally, we demonstrate how affordances of real objects can create synergies of auditory and visual feedback. As an application of the system, we built a spatial music mixing room.
TL;DR: In this article, the authors use Ambisonics to create dynamic spatial textures using granular synthesis and flocking algorithms, where only the overall motion and the motion of sounds relative to each other are perceived, while the individual trajectories followed by each grainboid are not generally clearly perceptible.
Abstract: and complex spatial designs can perhaps therefore be effective when used indirectly in this fashion. This approach is somewhat reminiscent of other abstract processes which have been used to indirectly create and control complex textures in orchestral music. Consider this quote by the composer Gyorgy Ligeti from an interview in 1978. “Technically speaking, I have always approached musical texture through partwriting. Both Atmospheres and Lontano have a dense canonic structure. But you cannot actually hear the polyphony, the canon. You hear a kind of impenetrable texture, something like a very densely woven cobweb. The polyphonic structure does not come through, you cannot hear it, it remains hidden in a microscopic, underwater world, to us inaudible [Bernard, 1987]. “ The dynamic spatial textures created using granular synthesis and flocking algorithms would seem to function in the same way, as in this case only the overall motion and the motion of sounds relative to each other will be perceived, while the individual trajectories followed by each grainboid are not generally clearly perceptible. In this case, it is the overall motion, rather than the specific location or direction, which is important and hence, Ambisonics would seem to be a suitable spatialization scheme for this purpose.
TL;DR: A new sound-projection system for multichannel loudspeaker setups that has been developed by the authors, based on the simulation of microphone techniques and acoustic enclosures that goes beyond the original concept by allowing strategic violations of physically possible parameters.
Abstract: This article describes a new sound-projection sys- tem for multichannel loudspeaker setups that has been developed by the authors. The system, called Virtual Microphone Control (ViMiC), is based on the simulation of microphone techniques and acoustic enclosures. In auditory virtual environments (AVEs), it is often required to position an anechoic point source in three-dimensional space. When sources in such applications are to be displayed using mul- tichannel loudspeaker reproduction systems, the processing is typically based upon simple amplitude- panning laws. With an adequate loudspeaker setup, this approach allows relatively accurate positioning of spatial images in the horizontal plane, but it lacks the flexibility many composers of computer music would like to have. This article describes an alternative approach based on an array of virtual mi- crophones. In the newly designed environment, the microphones, with adjustable directivity patterns and axis orientations, can be spatially placed as de- sired. Each virtual microphone signal is then fed to a separate (real) loudspeaker for sound projection. The system architecture was designed for a maximum flexibility in the creation of spatial imagery. Despite its flexibility, the system is intuitive to use because it is based on the geometrical and physical principles of microphone techniques. It is also consistent with the expectations of audio engineers to create sound imagery similar to that associated with standard sound-recording practice, but it goes beyond the original concept by allowing strategic violations of physically possible parameters; namely, new supernatural microphone directivity patterns can be implemented into the ViMiC system. This article begins with a review of various microphone techniques on which the ViMiC system Computer Music Journal, 32:3, pp. 55-71, Fall 2008 c � 2008 Massachusetts Institute of Technology. relies and alternative sound-projection techniques. Next, the fundamental physical concepts on which the ViMiC system is based are described. In the following section, software implementation of the system is outlined with a focus on strategies to keep processor load and system latency low. The article concludes with a description of several projects that involved the ViMiC system.
TL;DR: Brant's role in the development of "spatial music" in which sound placement and movement have structural and aesthetic function has been overshadowed by that of the European avant-garde, especially Karlheinz Stockhausen as mentioned in this paper.
Abstract: In 1933 Henry Cowell praised Henry Brant as "a musician with knowledge, technique, original ideas, feeling, something to say, and courage."' An overstatement, perhaps, in reference to a young composer, these words acquire a new significance over sixty years lateryears filled with explorations of Brant's idea that space is "an essential aspect of musical composition."2 Yet, the composer's role in the development of "spatial music" in which sound placement and movement have structural and aesthetic function, has been overshadowed by that of the European avant-garde, especially Karlheinz Stockhausen.3 Here, I will discuss Brant's role as the pioneer of spatial music and point out his involvement in the American experimental tradition which was initiated by Charles Ives.4 The importance of spatial music in Brant's oeuvre becomes obvious when browsing through the catalogs of his works-for example, the 1992 Rental Catalog of the publisher Carl Fisher lists seventy-six spatial pieces and fifty-seven nonspatial compositions. The range of examples cited here will be much more limited: references to four compositions will serve to highlight the basic features of Brant's genre of spatial music and help to trace the course of its development since his first spatial work, Antiphony I (1953).5 In Antiphony I the symphony orchestra is divided into five groups-
TL;DR: This work builds the equipment necessary to modify a standard electric guitar into a hexaphonic guitar, and presents an application of non-negative matrix factorisation to the task of transcription-where a basis for each note on the fretboard is learned and fitted to a magnitude spectrogram of thehexaphonic recording.
Abstract: Automatic music transcription is a widely studied problem, Typically, recordings that are used for transcription are taken from standard instruments, in the case of electric stringed instruments-such as the electric guitar-the recordings are captured from a standard pick-up, which unwantedly mixes the signals from each string and complicates subsequent analysis. We propose an approach to electric guitar transcription where the signal generated by each string at the guitar pickup is captured and analysed separately; thus providing six separate signals as opposed to one mixed signal, which enables finger positions to be identified. Such an instrument is known as a hexaphonic guitar and is a popular instrument for spatial music performances. We build the equipment necessary to modify a standard electric guitar into a hexaphonic guitar, and present an application of non-negative matrix factorisation to the task of transcription-where a basis for each note on the fretboard is learned and fitted to a magnitude spectrogram of the hexaphonic recording, which then undergoes a nonlinearity generating a piano roll representation of the music performance.