Parallel supercomputer architectures and their programming models

TL;DR: A taxonomy of the existing forms of parallel computer architectures, based on the characteristics of the hardware architecture and the abstract machine layered upon it, and their corresponding programming paradigms are characterized.

Abstract: The paper presents a taxonomy of the existing forms of parallel computer architectures, based on the characteristics of the hardware architecture and the abstract machine layered upon it. The abstract machine reflects the programming models provided. The main classes of hardware architectures are: physically shared memory systems and distributed memory systems. Distributed memory systems may be remote memory access architectures or message passing architectures. The major forms of abstract machine architecture are: message passing systems and logically shared memory architectures. Three solutions for logically shared memory architectures are known (1) distributed shared memory architectures, (2) multi-threaded architectures, and (3) virtual shared memory architectures, All three types are discussed in detail under the aspects of performance, programmability, and scalability, and their corresponding programming paradigms are characterized. The implications of the three concepts on node architecture and the requirements of latency minimization or latency hiding are discussed and illustrated by examples taken from pioneering realizations of the three kinds of architecture such as DASH, ∗T, and MANNA.