A spreadsheet model for using web service data

TL;DR: Gneiss is a tool that extends the familiar spreadsheet metaphor to support working with data returned from web services, and allows users to extract the desired fields in web service data using drag-and-drop, and refine the results through spreadsheet formulas, along with sorting and filtering the data.

Abstract: Web services offer a more reliable and efficient way to access online data than scraping web pages. However, web service data are often in complex hierarchical structures that make it difficult for people to extract the desired parts or to perform any further data manipulation without writing a significant amount of surprisingly intricate code. In this paper, we present Gneiss, a tool that extends the familiar spreadsheet metaphor to support working with data returned from web services. Gneiss allows users to extract the desired fields in web service data using drag-and-drop, and refine the results through spreadsheet formulas, along with sorting and filtering the data. Hierarchical data are stored as nested tables in the spreadsheet and can be flattened for future operations. Data flow is two-way between the spreadsheet and the web services, enabling people to easily make a new request by modifying spreadsheet cells. In addition, using the dependency between spreadsheet cells, our tool is able to create parallel-running data extractions based on the user's sequential demonstration. We use a set of examples to demonstrate our tool's ability to create fast and reusable data extraction and manipulation programs that work with complex web service data.