Intermediate Notation for Provenance and Workflow Reproducibility

Abstract: . The automation of data analysis in the form of scientific workflows is a widely adopted practice in many fields of research nowadays. Computationally driven data-intensive experiments using workflows enable Automation, Scaling, Adaption and Provenance support (ASAP). However, there are still several challenges associated with the effective sharing, publication, un-derstandability and reproducibility of such workflows due to the incomplete capture of provenance and the dependence on particular technical (software) platforms. This paper presents CWL-Prov, an approach for retrospective provenance capture utilizing open source community-driven standards involving application and customization of workflow-centric Research Objects (ROs). The ROs are produced as an output of a workflow enactment defined in the Common Workflow Language (CWL) using the CWL reference implementation and its data structures. The approach aggregates and annotates all the resources involved in the scientific investigation including inputs, outputs, workflow specification, command line tool specifications and input parameter settings. The resources are linked within the RO to enable re-enactment of an analysis without depending on external resources. The workflow provenance profile is represented in W3C recommended standard PROV-N and PROV-JSON format to capture retrospective provenance of the workflow enactment. The workflow-centric RO produced as an output of a CWL workflow en-actment is expected to be interoperable, reusable, shareable and portable across different platforms. This paper describes the need and motivation for CWLProv and the lessons learned in applying it for ROs using CWL in the bioinformatics domain. Research

TL;DR: This paper proposes a PROV extension to represent mutable data structures by adding reference derivations and checkpoints and results indicate a reduction in the storage overhead for assignments and changes in data structures.

TL;DR: Details about the provenance RDataTracker collects and the mechanisms used to collect it are provided and it is speculated about how this rich source of information could be used by other tools to help an R programmer gain a deeper understanding of the software used and to support reproducibility.

TL;DR: The state of the art in provenance for scripts is surveyed by following an exhaustive protocol of forward and backward literature snowballing and a taxonomy is proposed and classified the approaches using this taxonomy.

TL;DR: An update to the taverna tool suite is provided, highlighting new features and developments in the workbench and the Taverna Server.

TL;DR: The VisTrails system represents the initial attempt to improve the scientific discovery process and reduce the time to insight, and is presented by presenting actual scenarios in which scientific visualization is used and showing how the system improves usability, enables reproducibility, and greatly reduces the time required to create scientific visualizations.

TL;DR: The authors give an overview of important concepts related to provenance management, so that potential users can make informed decisions when selecting or designing a provenance solution.

TL;DR: This document introduces the provenance concepts found in PROV and defines PROV-DM types and relations.