Workflow Interoperability in Grid-based Systems

TL;DR: The main requirements for developing and using cross-origination grid-based workflows for engineering applications and product design are discussed and some of the higher-level issues and problems that exist when multiple workflow systems exist within the same environment are described.

Abstract: The use of workflow technology within grid-based computing has received a lot of attention recently. Informally, a workflow is an abstract description of the steps or tasks required for executing a particular real-world process, and the flow of information between these tasks. Work passes through the flow from start to finish and activities might be executed by people or by system functions. Within a grid computing environment, a workflow provides the mechanisms for constructing distributed end user applications through the composition of distributed data services and computational services. Workflow systems typically enable users to construct their workflows using a visual interface and to submit them for execution by an execution engine that controls the invocation and data transfer between the different services. Within large collaborative projects, it is essential to admit the existence of multiple workflow systems that are already in use by different partners and the fact these tools are used for addressing different requirements in different organizations. A clear example can be seen in the EU-funded SIMDAT project which involves a consortium of 29 research and industrial partners. The project is developing grid-based technology for enabling large-scale industrial product design and focuses on four application areas: product design in the automotive, aerospace and pharmaceutical industry as well as service provision in meteorology. Within SIMDAT, different industrial and academic partners have in place their own workflow solutions. These range from hard-coded workflows written using scripts to using one of three workflow engines (Taverna/Freefluo, InforSense KDE and LMS Optimums). Although these workflow systems may look superficially similar, each has been designed and optimized for different requirements. Rather than attempting to enforce a single workflow system on all partners, we foster an approach that admits such heterogeneity and enabling run-time interoperability between these different workflows. In this paper, based on our experience within the SIMDAT project, we discuss the main requirements for developing and using cross-origination grid-based workflows for engineering applications and product design. Our examples are based on the automotive, aerospace and pharmaceutical sectors and are used to provide comparison between the different workflow systems used within the project. We also describe how the different workflow systems used within the project have been modified to coordinate the execution of remote grid services in addition to remote web services. Our experience here is based on using GRIA as a middleware for supporting Grid service composition. We then describe our experience in developing and using a generic approach for achieving interoperability between the workflow systems across-organizational boundaries. The conceptual approach is generic and enables us to wrap remote services as grid services and also to wrap remote workflow engines as services to which workflow descriptions can be submitted. Finally, we describe some of the higher-level issues and problems that exist when multiple workflow systems exist within the same environment and provide suggestions for addressing them.