Interactive data exploration with a supercomputer

TL;DR: This paper describes an experiment in exploratory data visualization using a massively parallel processor that was able to find new features in some of the authors' oldest datasets and to create more vivid presentations of familiar features in these datasets.

Abstract: Scientific data of high-dimensionality is particularly difficult to understand if there are complex (and perhaps unknown) interactions among parameters. In such situations, the scientist can turn to a process Tukey has called exploratory data visualization. In exploratory data visualisation, we typically do not know exactly what we are looking for; instead, we explore the data with a variety of visualization techniques that can help us understand the nature of the data by demonstrating patterns in it. Given the complexity of the data and the large number of ways to map the data to a display, exploratory visualization can be truly effective only if it can also be interactive. In this paper we describe an experiment in exploratory data visualization using a massively parallel processor. With this approach we were able to find new features in some of our oldest datasets and to create more vivid presentations of familiar features in these datasets. Our experience has also led us to a better understanding of the nature of exploratory visualization and has resulted in some formal representations of the interaction process in this environment.