Vision and Art: The Biology of Seeing

Roy Howat’s Debussy in Proportion: A MusicaI Analysis is written with rigour, precision and lucidity. From the outset he states the primary purpose for writing the book to be ‘the discovery that much of Debussy’s music contains intricate proportional systems which can account both for the precise nature of the music’s unorthodox forms and for the difficulty in defining them in more familiar terms’. Two aspects of Debussy’s music, not usually highlighted, are brought analytically into the foreground for proportional treatment by Howat: the dynamic shape, and the dramatic shape of the music. Dynamic shape is determined by measuring the distance between dynamic focal points, usually the peaks. Dramatic shape refers to the tonal organisation of the music. It is determined by the distance between the points at which the music changes tonal direction — this includes chromatic and modal changes. A third aspect measures for proportionality is the length of thematic sections. Howat reveals that these thre...

Debussy in proportion: A musical analysis

This paper introduces strategies for the electroacoustic community to relate to, and engage with, the visual music phenomenon. It addresses technological, historical, cultural and idiomatic intersections between the two art forms. From the personal viewpoint of a trained synaesthete of acousmatic origin, the boundaries between sonic and audiovisual compositional practices appear somewhat porous. The electroacoustic language is intrinsically visual, even within its acousmatic paradigm. Visible morphologies acquire a sonorous dimension as soon as we uproot them from their cinematographic habitat and plunge them into the cauldron of a new alchemy. Multi-disciplinary lines of enquiry are essential to elucidate the workings of complex multimediatic interactions such as those at play in visual music. Yet, a holistic view of the creative and technological pathways is equally significant so that artistic truths, and myths, can be (re)discovered amidst lines of code or loops of wire connecting our computer peripherals. Thus this article is written both with the language of an analyst and, perhaps more, with the expressions and idiosyncrasies of an academic composer. A few selected examples from the contemporary repertoire are discussed to exemplify a variety of approaches to visual music composition, including extracts from the work Patah (Garro 2010).

From sonic art to visual music: Divergences, convergences, intersections

This research explores music in space, as experienced through performing and music-making with interactive systems. It explores how musical parameters may be presented spatially and displayed visually with a view to their exploration by a musician during performance. Spatial arrangements of musical components, especially pitches and harmonies, have been widely studied in the literature, but the current capabilities of interactive systems allow the improvisational exploration of these musical spaces as part of a performance practice. This research focuses on quantised spatial organisation of musical parameters that can be categorised as grid music systems (GMSs), and interactive music systems based on them. The research explores and surveys existing and historical uses of GMSs, and develops and demonstrates the use of a novel grid music system designed for whole body interaction. Grid music systems provide plotting of spatialised input to construct patterned music on a two-dimensional grid layout. GMSs are navigated to construct a sequence of parametric steps, for example a series of pitches, rhythmic values, a chord sequence, or terraced dynamic steps. While they are conceptually simple when only controlling one musical dimension, grid systems may be layered to enable complex and satisfying musical results. These systems have proved a viable, effective, accessible and engaging means of music-making for the general user as well as the musician. GMSs have been widely used in electronic and digital music technologies, where they have generally been applied to small portable devices and software systems such as step sequencers and drum machines. This research shows that by scaling up a grid music system, music-making and musical improvisation are enhanced, gaining several advantages: (1) Full body location becomes the spatial input to the grid. The system becomes a partially immersive one in four related ways: spatially, graphically, sonically and musically. (2) Detection of body location by tracking enables hands-free operation, thereby allowing the playing of a musical instrument in addition to “playing” the grid system. (3) Visual information regarding musical parameters may be enhanced so that the performer may fully engage with existing spatial knowledge of musical materials. The result is that existing spatial knowledge is overlaid on, and combined with, music-space. Music-space is a new concept produced by the research, and is similar to notions of other musical spaces including soundscape, acoustic space, Smalley's “circumspace” and “immersive space” (2007, 48-52), and Lotis's “ambiophony” (2003), but is rather more textural and “alive”—and therefore very conducive to interaction. Music-space is that space occupied by music, set within normal space, which may be perceived by a person located within, or moving around in that space. Music-space has a perceivable “texture” made of tensions and relaxations, and contains spatial patterns of these formed by musical elements such as notes, harmonies, and sounds, changing over time. The music may be performed by live musicians, created electronically, or be prerecorded. Large-scale GMSs have the capability not only to interactively display musical information as music representative space, but to allow music-space to co-exist with it. Moving around the grid, the performer may interact in real time with musical materials in music-space, as they form over squares or move in paths. Additionally he/she may sense the textural matrix of the music-space while being immersed in surround sound covering the grid. The HarmonyGrid is a new computer-based interactive performance system developed during this research that provides a generative music-making system intended to accompany, or play along with, an improvising musician. This large-scale GMS employs full-body motion tracking over a projected grid. Playing with the system creates an enhanced performance employing live interactive music, along with graphical and spatial activity. Although one other experimental system provides certain aspects of immersive music-making, currently only the HarmonyGrid provides an environment to explore and experience music-space in a GMS.

The HarmonyGrid : music, space and performance in grid music systems

Visualizers are usually added into music players to augment the listening experiences of hearing users. However, for the case of most members of the Deaf and Hard of Hearing (DHH) community, they have partial deafness which may give them a "limited" listening experience as compared to their counterparts. In this paper, we present ViTune, a visualizer tool that enhances the musical experiences of the DHH through the use of an on-screen visualizer generating effects alongside music. We observed how members of the DHH community typically experienced music through an initial user study. We then iterated on developing a visualizer prototype where we did multiple usability tests involving at least 15 participants from the DHH community. We observed how they experienced music with the help of our prototype and noticed that certain music features and elements augmented these experiences. Visualization attributes, their matching qualitative descriptions and equivalent subjective attributes were identified with the help of music experts. Also, affects hypothesized to be induced and dissuaded were identified in improving these listening experiences.

ViTune: A Visualizer Tool to Allow the Deaf and Hard of Hearing to See Music With Their Eyes

animation, often called visual music, can have a structural base similar to that of absolute music. In this article, I propose a groundwork for a practical theory of visual music composition. The approach builds from the simple premise that the resolution of tension moves us through time. The article will review some fundamentals from art and design and apply them to temporal structure. To establish the idea of visual consonance, the design principle of proportion will be presented. From this, I will develop and codify the idea that it is possible to resolve visual dissonance to consonance, and so move a viewer through time in a way similar to tonal harmony in music. We will then briefly consider problems of color. The article will review some traditional approaches to color harmony and suggest a simple hierarchical approach to working with color in time. Expanding to a more comprehensive system of time-based design, the writings of Russian filmmaker Serge Eisenstein will be mined for raw material (Eisenstein 1942, 1949). Eisenstein believed that “art is always conflict” (Eisenstein 1949, p. 46). For this article, conflict is defined as the opposition of forces that motivates and shapes action. The opposed forces are dissonance/consonance or tension/ release. Eisenstein’s writings contain many ideas that are useful in the development of a theory of visual music—especially his definition of cinematic montage. From the concepts of montage and timedesign, a tentative but useable grammar for structuring time-based visual art can be developed and eventually applied to the integration of sound and image. For this article, I created some simple video examples as illustrations. Where useful, I will also refer to the works of some visual music pioneers and other filmmakers. (Pieces were selected based on relevance and availability, and the list is by no means exhaustive.) Visual Music and Its History Visual music can be defined as time-based visual imagery that establishes a temporal architecture in a way similar to absolute music. It is typically nonnarrative and non-representational (although it need not be either). Visual music can be accompanied by sound but can also be silent. One can imagine the origins of visual music going back to the discovery of fire and dancing shadows on cave walls, or perhaps reflections of clouds when first seen on the surface of rippling water. In more recent history, color organs (instruments that projected colored light) were seen by the public as early as the 18th century with Castel’s Ocular Harpsichord. In the 1920s Thomas Wilfred toured the United States and Europe performing on his Clavilux, an early electrical instrument that created clouds and streams of continuous color. This history intensified over the last century, parallel with the development of cinema. While the Hollywood-style narrative dominated (and dominates) cinema, pioneering filmmakers were working to develop a non-narrative language of light. Typically non-academics, these pioneers worked in an experimental tradition similar to that of music composers such as Eric Satie, John Cage, and Karlheinz Stockhausen. Among the early filmmakers were Germans Walter Ruttman, Viking Eggeling, and Hans Richter. Oskar Fischinger, also from Germany and moving later to the United States, worked over thirty years creating abstract Computer Music Journal, 29:4, pp. 11–24, Winter 2005 © 2005 Massachusetts Institute of Technology. animations (Mowitz 2004). The career of New Zealander Len Lye, working mainly in London, spanned decades. American pioneers included John and James Whitney, Mary Ellen Butte, Stan Brakhage, and Jordan Belson. Norman McClaren, Evelyn Lambart, and their colleagues at the National Film Board of Canada also created significant works. There were many others (Russet and Starr 1976). They all struggled against financial, institutional, and technical barriers, yet they left a body of work that provides a base from which to construct a theory of visual music. This work is finding renewed interest as evidenced by recent exhibitions. The 2003 Sonic Light Festival in Amsterdam featured many of the pioneering animations, as did Sons & Lumieres, Une histoire du son dans l’art du 20 siecle at the Centre Pompidou in Paris, closing in January 2005 (Cruse 2004; see also www.sonicacts.com/03). Also in 2005, the Los Angeles Museum of Contemporary Art opened the exhibit Visual Music: Synaesthesia in Art and Music Since 1900, which then moved to the Hirshorn Museum in Washington, D.C. (Brougher et al. 2005). Much of this work is becoming easier to find on video and DVD at online sources such as the Iota Center (www.iotacenter.org), the Center for Visual Music (centerforvisualmusic.org), and even Amazon.com. Visual Music/Absolute Music The concept of absolute music can be constructed from Igor Stravinsky’s statement, “Music means nothing outside itself” (Stravinsky 1956). This illustrates a common mindset held by many composers for centuries. Form and content were often one in instrumental music. (This of course does not include programmatic music and especially the occasional attempts to imitate real-world sounds, such as bird calls, thunderstorms, etc.) Today, acoustic and electronic instruments can be used as generators of abstract sounds without a referent in the real world beyond the instruments themselves. These nonreferential sounds are used to create abstract temporal structures, from traditional musical forms to those of a more experimental nature. (Many of these new forms also find correspondences in visual music.) A simple definition of absolute music is the structuring of time with the materials of sound patterns (excluding again program music and literary forms such as song, opera, and theater). Stravinsky’s comment refers to music composition as the making of “art for art’s sake.” A musical work seeks from the listener an aesthetic response to the perception of sonic pattern—the appreciation of “significant form.” This was a primary focus of modernism and its formalist leanings (Bell 1914). When visual artists discuss composition, they are generally referring to static design, the formal distribution of objects in the picture plane or in threedimensional space. (This article will deal exclusively with two-dimensional space.) In the 20th century, artists sought to express design in the abstract—significant form in visual space. They began to consider seriously their work in a musical way, and the abstract form of the work was the content. Kandinsky, considered by many to be the father of abstract painting, said: A painter . . . in his longing to express his inner life cannot but envy the ease with which music, the most non-material of the arts today, achieves this end. He naturally seeks to apply the methods of music to his own art. And from these results that modern desire for rhythm in painting, mathematical, abstract construction, for repeated notes of color, for setting color in motion. (Kandinsky 1914) Using music as a basis for visual work was a concept explored by many artists in the early 20th century. Notable is the work by Serge Eisenstein, a Russian avant-garde filmmaker who put these ideas into practice and, fortunately for us, wrote down many of his theories of cinema. These ideas still have potency and are especially relevant in discussions of visual music, time design, and the combining of sound and image.

https://www.jstor.org/stable/pdfplus/3681478.pdf

Foundations of a Visual Music

Musical connections and heterophonic maps

Time slices to sonic maps

Data mapping is the essence of being human. This report follows the process of data mapping; the transmission of a structured utterance from one domain to another. It starts with signals in the world and moves through experience and engagement with the world through those signals. The paper develops descriptions of the materials and mechanisms of data mapping. The descriptions are aids and conveniences in the effort to understand the systemic workings of the process of communication. From a systems perspective one might find points of leverage in their own involvement with the processes of communication and data mapping. Recognizing these leverage points can help in activities of art making, information design and in simply living.

Materials of the Data Map

We are order-seeking creatures. Through our senses we receive signals from the world around us. Our senses convert these signals into electrical data we then interpret, thus creating all of our experience. We try to make sense of these signals. What endures? What repeats? What changes? We look for structure. We recognize and compare patterns, trying to understand the sensory data. We build a basis from which we make choices. Living is the making of choices, based upon the receiving of signals and perceiving pattern within those signals. Synaesthesia is a re-mapping of sensory data. Signals coming though one sense organ get rerouted in the brain and are received as if coming through a different sense organ. Inside the brain all sensory data are essentially the same—neural, electrical impulses. The materials of the sense data are the same, but depending on the sense organ that receives the original signal, the data are processed differently. Sometimes these signals get crossed. Sound, for example might enter the ear, but once in the brain the sensory data travels down the visual pathways and color is seen. [Livingstone 2002] With other routings one might taste a whisper or smell a tickle. If neural data takes a detour then experience, while synchronized with the external source event, is mapped and perceived atypically (but not necessarily uninterestingly). Some people are born with these detours built into their neural roadways. Most of us do not experience this at all. Some of us try to simulate it. We make art. Artists are order-making creatures. Sometimes we want to simply understand what order is. What establishes order? What defines pattern? Digital artists have number as their medium. Like homogeneous neural data, all digital representations of signals are in the same format. Digital data are binary numbers. As we create a digital experience it exists as numbers and we can map those numbers freely. If we use numbers to create aural experience we make a sonic map. A visual experience maps numbers into colors, making an image. It follows that we can create a sonic map of the numbers from the image and hear it. Of course the reverse is true. Monet's Water Lilies can be a symphony, Beethoven's Ode to Joy a wash of pigment on canvas. Synaesthesia is easy in a digital world. In mapping numbers into experience, aesthetic decisions are made. What palette of colors to use? …

Synaesthesia, data mapping and synchronicity

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Brian Evans

Author Tools

Chat about Author