Heterogeneous Multithreaded Computing

TL;DR: This thesis implements a heterogeneous multithreaded runtime system, and presents an implementation of Cilk that supports execution in heterogeneous environments, and proves its efficiency.

Abstract: This thesis studies issues introduced by supporting multithreaded computing in environments composed of workers heterogeneous with respect to data representation, speed, and scale. We implement a heterogeneous multithreaded runtime system, and prove its efficiency. We present an implementation of Cilk that supports execution in heterogeneous environments. To support communication between workers with different data representations, we propose two mechanisms. The first mechanism translates messages to a canonical form on transmission and to the native host format on reception. The second mechanism generates signature tables of structures that are communicated between the workers to guide translation. We propose but do not implement an extension to generate signature tables for all C functions so that functions can be passed as messages. We prove the following: for a fully strict Cilk program with T1 total work and a critical path of length To, running on P workers with a maximum communication time amax, average performance 7rave, and maximum performance rmax, we obtain the execution time Tp = O(TI17aveP + TYmaxToormaxllrave) Furth rmore, the "effective parallelism" of such a computation is only a factor of omax/rmax worse than the average parallelism T1/Too. Thesis Supervisor: Charles E. Leiserson Title: Professor, Massachusetts Institute of Technology