
 Surprise Machines is a project of experimental museology that
 sets out to visualize the entire image collection of the Harvard Art Museums,
 with a view to opening up unexpected vistas on more than 200,000 objects usually
 inaccessible to visitors. The project is part of the exhibition organized by
 metaLAB (at) Harvard entitled Curatorial A(i)gents and explores the limits of
 artificial intelligence to display a large set of images and create surprise
 among visitors. To achieve this feeling of surprise, a choreographic interface
 was designed to connect the audience’s movement with several unique views of the
 collection.

Surprise machines

Audiovisual (AV) archives are invaluable for holistically preserving the past. Unlike other forms, AV archives can be difficult to explore. This is not only because of its complex modality and sheer volume but also the lack of appropriate interfaces beyond keyword search. The recent rise in text-to-video retrieval tasks in computer science opens the gate to accessing AV content more naturally and semantically, able to map natural language descriptive sentences to matching videos. However, applications of this model are rarely seen. The contribution of this work is threefold. First, working with RTS (Télévision Suisse Romande), we identified the key blockers in a real archive for implementing such models. We built a functioning pipeline for encoding raw archive videos to the text-to-video feature vectors. Second, we designed and verified a method to encode and retrieve videos using emotionally abundant descriptions not supported in the original model. Third, we proposed an initial prototype for immersive and interactive exploration of AV archives in a latent space based on the previously mentioned encoding of videos. 

Latent Wander: an Alternative Interface for Interactive and Serendipitous Discovery of Large AV Archives

We propose a half-day workshop at IEEE VIS 2023 on the topic of communication and collaboration around data. Speciﬁcally, we aim to gather researchers interested on multimodal, synchronous, and remote or hybrid forms of communication and collaboration within organizational and educational settings. This topic lies at the intersection of data visualization, human-computer interaction

A Workshop on New Multimodal Experiences for Remote Communication and Collaboration Around Data

The proliferation of data throughout society has produced an increased interest in novel ways to represent information to provide new insights, re-humanize data, or allow broader participation. Many novel data representations allow people to physically interact with information by picking up, touching, and manipulating artefacts that encode data through their material properties. In this paper, we draw on the arts to explore how we might design for deeper physical engagement with data. How can we feel data through movements, instead of through objects? We examine nine instructional movement sequences created by five dancers as part of a larger workshop on data and dance. Our findings shed light on promising design strategies for somatically-focused data representations including observations on the relational, layered, and emotionally engaged approaches taken by dancers. We also reflect on these approaches in relation to soma design and identify areas for future work in "data somatics."

Feeling Data through Movement: Designing Somatic Data Experiences with Dancers

Abstract The use of digital museum objects has become an essential part of museums' communication and marketing strategies, research and teaching, and curatorial practices. This new visibility, amplified by the COVID‐19 crisis, has not only revealed the possibilities of digital museum objects but has also underscored significant challenges, including the intricate relationship between digital museum objects and physical objects, the impact of digital museum objects on knowledge creation, and the online interaction with art and cultural heritage. Furthermore, it has drawn attention to the digital platforms that host digital museum objects, ranging from museum collection databases to online encyclopedias, cultural heritage platforms, and social media. The present paper explores these issues by examining how digital platforms are changing the way digital reproductions of artworks are used and reimagined, both inside and outside the institutions that house the artworks. To illuminate these dynamics, it looks at specific case studies, including the Getty Challenge, the online circulation of Delacroix's La liberté guidant le peuple , and the collection databases of Belgian museums and of the Mauritshuis. Theoretically, it combines the art historical concept of circulation with the notion of gray and colored memory drawn from digital memory studies. In doing so, it conceives of museum databases and cultural heritage platforms, as spaces of participation and neglect, of memory and oblivion, with a vast potential for producing new perspectives on art and cultural heritage and telling new (art) histories. In conclusion, the paper advocates for “circulation” as a key concept for revitalizing online collections of digital reproductions of artworks. 

Art in circulation: Creating content and context for digital reproduction of artworks

In 2019, metaLAB (at) Harvard began work on Curatorial A(i)gents, a digital exhibition that was slated to premiere at the Harvard Art Museums’ Lightbox Gallery in 2020. Half of the projects would be interactive, using mouse and keyboard conventions. With the advent of Covid-19 and the postponement of the show, the authors set out to develop an interface solution that would enable visitors to interact with the works without having to touch any public devices like a tablet. Toward this end, we prototyped a “choreographic interface” that uses machine vision and machine learning to interpret a full-torso gestural vocabulary, which is then translated into interactions. To make the choreographic interface, we relied on open-source solutions, which have all come with equal limitations and opportunities. In 2022, Curatorial A(i)gents was presented in the Lightbox Gallery, where we had the opportunity to test and demonstrate the interface. This paper discusses our design journey in making a choreographic interface using open-source technologies during Covid-19.

Designing a Choreographic Interface During COVID-19

ABSTRACT The motivation to make information more accessible in data communication – physically to the senses and culturally to different audiences – has led designers to experiment, for example, in data physicalization and data sonification. Experimentation overlapping data communication and the performance arts suggests another practice is emerging. Calling it data embodiment, I define the term as a communicative and artistic method that approaches the body as a choreographic medium to physicalize and perform abstract data. The intent of this paper is to sufficiently delineate the space where bodily movement and abstract data have begun to intermingle across several milieu under the concept. The paper begins by contextualizing data embodiment in design, then continues by observing how choreographers have previously approached dance as data, and finishes with a case study where an environmental migration dataset is choreographed. A brief section on human-centered datasets and the surveillant technologies that produce them peppers the overarching discussion on bodily movement and abstract data. By reconstituting data in time and space, especially in the context of performance, otherwise sterile datasets are shown to become more physically and culturally accessible as well as aesthetically expressive.

Data embodiment: approaching the body as a choreographic medium for performing abstract data

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Lins Derry

Author Tools

Chat about Author

Papers

Surprise machines

Designing a Choreographic Interface During COVID-19

Data embodiment: approaching the body as a choreographic medium for performing abstract data