We survey parallel programming models and languages using six criteria to assess their suitability for realistic portable parallel programming. We argue that an ideal model should by easy to program, should have a software development methodology, should be architecture-independent, should be easy to understand, should guarantee performance, and should provide accurate information about the cost of programs. These criteria reflect our belief that developments in parallelism must be driven by a parallel software industry based on portability and efficiency. We consider programming models in six categories, depending on the level of abstraction they provide. Those that are very abstract conceal even the presence of parallelism at the software level. Such models make software easy to build and port, but efficient and predictable performance is usually hard to achieve. At the other end of the spectrum, low-level models make all of the messy issues of parallel programming explicit (how many threads, how to place them, how to express communication, and how to schedule communication), so that software is hard to build and not very portable, but is usually efficient. Most recent models are near the center of this spectrum, exploring the best tradeoffs between expressiveness and performance. A few models have achieved both abstractness and efficiency. Both kinds of models raise the possibility of parallelism as part of the mainstream of computing.

/pdf/models-and-languages-for-parallel-computation-1sticvkm4u.pdf

Models and languages for parallel computation

Data parallelism is often seen as a form of explicit parallelism for SIMD and vector machines, and data parallel programming as an explicit programming paradigm for these architectures. Data parallel languages possess certain software qualities as well, which justifies their use in higher level programming and specification closer to the algorithm domain. Thus, it is interesting to study how the data parallel paradigm can be best realized in a declarative setting, since declarative languages offer a pure view of computation which is good for these purposes. For numerical computing the functional programming paradigm is especially attractive, since numerical algorithms often are specified by recursion equations and thus can be translated more or less directly into recursive functional programs. Merging the data parallel and functional paradigms then yields languages and formalisms where many algorithms can be expressed in a very succinct fashion. In this paper we review data parallelism, functional programming, and existing approaches to the integration of the two paradigms. We then proceed to describe a formalism for data parallel functional programming, allowing very simple languages, where the view of aggregate data is particularly abstract. We explain how various data parallel operations can be expressed in this formalism. Finally, we conclude with a discussion of issues for languages based directly on the formalism.

Data Parallelism and Functional Programming

Most automatic parallelizes are based on the detection of independent operations. Dependence analysis is mainly a syntactical process, in which the actual data transformations are ignored. There is another source of parallelism, which relies on semantical information, namely the detection of reductions and scans. Scans and reductions are quite frequent in scientific codes and are implemented efficiently on most parallel computers. We present here a new Scan detector which is based on the normalization of systems of recurrence equations. This allows the detection of scans in loops nests of arbitrary depth and on multi-dimensional arrays, and gives a uniform treatment for scalar reductions, array reductions, and arrays of reductions.

Detection of scans

Concevoir et implanter des programmes paralleles est une tâche complexe, sujette aux erreurs. La verification des programmes paralleles est egalement plus difficile que celle des programmes sequentiels. Pour permettre le developpement et la preuve de correction de programmes paralleles, nous proposons de combiner le langage parallele fonctionnel quasi-synchrone BSML, les squelettes algorithmiques - qui sont des fonctions d’ordre superieur sur des structures de donnees reparties offrant une abstraction du parallelisme – et l’assistant de preuve Coq, dont le langage de specification est suffisamment riche pour ecrire des programmes fonctionnels purs et leurs proprietes. Nous proposons un plongement des primitives BSML dans la logique de Coq sous une forme modulaire adaptee a l’extraction de programmes. Ainsi, nous pouvons ecrire dans Coq des programmes BSML, raisonner dessus, puis les extraire et les executer en parallele. Pour faciliter le raisonnement sur ceux-ci, nous formalisons le lien entre programmes paralleles, manipulant des structures de donnees distribuees, et les specifications, manipulant des structures sequentielles. Nous prouvons ainsi la correction d’une implantation du squelette algorithmique BH, un squelette adapte au traitement de listes reparties dans le modele de parallelisme quasi synchrone. Pour un ensemble d’applications partant d’une specification d’un probleme sous forme d’un programme sequentiel simple, nous derivons une instance de nos squelettes, puis nous extrayons un programme BSML avant de l’executer sur des machines paralleles.

Environnement pour le développement et la preuve de correction systèmatiques de programmes parallèles fonctionnels

A lot of programming models have been proposed to deal with parallelism in order to express program transformations and refinements. This justifies to introduce an unifying theory to abstract different concepts. The aim of this paper is to introduce such a theory. This theory includes the definitions of problems, programs and transformation rules. It is founded on the simple mathematical concepts of multiset and of an equivalence between their representations as data fields. Program transformations are founded on this equivalence and defined from a refinement relation. Due to the unifying aspect of this theory, solutions that can be reached by these transformations are relevant to various synchronous or asynchronous computing models.

PEI: a Single Unifying Model to Design Parallel Programs

The aim of this paper is to introduce a new modeling, using symbols and functional notations, called the language PEI (as Parallel Equations Interpretor), which could unify the classical approaches of program parallelization, according to the sorts of problems and the target computation schemes. Due to its fundamental structure, disconnected from concrete drawings, this modeling offers a straightforward generalization to adress convex or non-convex computation domains, synchronous or asynchronous computations. This sort of programming formalism allows a powerful structuration of statements and a stepwise tranformation technique based on a semantical equivalence definition and a refinement calculus. From initial problem statements, transformations of expressions lead to various definition structures, which can match given computation models.

PEI: A language and its refinement calculus for parallel programming☆

This paper presents the main concepts of the mathematical theory PEI for parallel programming and emphasizes its derivation power. The mathematical basis of this theory leads to a nice implementation in CENTAUR7 of an environment whose purpose is to transform parallel programs. It is illustrated by two similar examples: the convolution sum and the Dirichlet product. The second one uses non-affine dependencies that can be easily taken into account using PEI.

A mathematical theory and its environment for parallel programming

Reduction is one of the major issues in data parallel languages: it can be defined as a rule of program refinement. This article presents a theoretical framework, called Pei, the foundation of a formalism for parallel programming, where this rule can easily be expressed and applied. This formalism is founded on a small but powerful set of primitives: they are three operations on data fields and inverse operations. They induce a clear refinement calculus to transform specifications in executable programs by ensuring a safe process of design or optimization. We show how this approach allows to generalize the classical notion of reduction, by introducing a geometrical reduction and a functional one.

Reduction in PEI

We developed a theory in order to address crucial questions of program design methodology. This theory deals with data locality which is a main issue in parallel programming. In this article, we regard this theory and its model as a minimum semantic domain for data parallel languages. The introduction of a semantic domain is justified because the classical data parallel languages (HPF and C H ) have different intuitive semantics: Indeed, they use different concepts in order to express data locality. These concepts are alignment in HPF and shape in C H . Consequently these two languages define their own balance between compiler and programmer investments in order to reach program efficiency. We present our theory as a foundation for defining a better balance. � 2003 Elsevier B.V. All rights reserved.

/pdf/a-semantic-framework-to-address-data-locality-in-data-2q4r02usw4.pdf

A semantic framework to address data locality in data parallel languages

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Eric Violard

Author Tools

Chat about Author