There has been a great interest recently in systems that use graphics to aid in the programming, debugging, and understanding of computer programs. The terms “Visual Programming” and “Program Visualization” have been applied to these systems. Also, there has been a renewed interest in using examples to help alleviate the complexity of programming. This technique is called “Programming by Example.” This paper attempts to provide more meaning to these terms by giving precise definitions, and then uses these definitions to classify existing systems into a taxonomy. A number of common unsolved problems with most of these systems are also listed.

Visual programming, programming by example, and program visualization: a taxonomy

Syntax Graph (ASG) Derivation Spatial Relations Graph (SRG) Derivation translation syntax-directed editor interpreter/compiler layout algorithm graphics editor scanning rendering parsing Figure 3.1: Three levels of visual sentence representation. Please note that the derivation histories of SRGs and ASGs have to be modeled as separate data structures. A usual syntax-directed editor for ASGs or SRGs manipulates thereby the explicitly represented derivation history of a graph together with the graph itself. We will see later on in Section 3.4 that it is often possible to replace the graph grammar de nition of a visual language by a graph transformation system, which de nes all legal syntax-directed editor operations for a language of graphs without maintaining any derivation histories. It is then no longer possible to manipulate SRGs using a low-level graphics editor and to use a parser for regenerating the corresponding derivation history as well as the related ASG as shown in Figure 3.2. Abstract Syntax Graph (ASG) Spatial Relations Graph (SRG) translation structure-oriented editor interpreter layout algorithm layout editor rendering compiler Figure 3.2: A simpli ed visual sentence representation approach. Until now, we did not explain how ASGs and SRGs are related to each other, i.e., how an update of one graph is translated into an update of the related 120 CHAPTER 3. APPLICATION TO VISUAL LANGUAGES graph. In the most general case the relationships between ASGs and SRGs are modeled as coupled graph grammars as introduced in [80,93]. Assuming that any ASG graph grammar production is related to a single SRG graph grammar production and vice versa, it is a straightforward task to keep the ASG and SRG derivation histories and thereby the underlying graphs in a consistent state. In the absence of explicitly represented derivation histories a more primitive procedure has to be used to translate directly any ASG or SRG update into an update of the corresponding graph (as sketched in Figure 3.2.) The following subsections explain in more detail how graph grammars may be used to de ne the syntax of visual sentences in order to distinguish correct visual sentences from incorrect ones and to de ne their structure. Graph grammars are brie y introduced in the following Subsection 3.3.2. Subsections 3.3.3 and 3.3.4 then present two di erent approaches how to represent visual sentences by graphs. Subsection 3.3.3 uses hypergraphs, whereas Subsection 3.3.4 uses graph structures. For each of these approaches, di erent types of graph grammars used for describing VL syntax are discussed, too. These subsections as well as the following Section 3.5 about graph grammar parsers disregard the distinction between ASGs and SRGs to a certain extent. The discussion related to Figure 3.1 and Figure 3.2 will be resumed later on in Section 3.4. There we will see that all presented graph-based VL environment (editor) generators realize only subsets of the conceptual VL representation model of Figure 3.1. 3.3.2 Graph Grammars A visual language with a set of valid visual sentences that are represented by some kind of graphs (hypergraphs in Subsection 3.3.3 and graph structures in Subsection 3.3.4) corresponds with the language of the visual sentences' representations. Now we will brie y introduce general graph and hypergraph grammars. In the following subsections, we will then de ne several classes of grammars|context-free (CF) hypergraph grammars, context-free hypergraph grammars with embeddings (CFe), and unrestricted graph grammars|which are apt to de ne such languages of graph representations. For these grammars, we will present parsing algorithms in Section 3.5. Such a parser is necessary for distinguishing correct visual sentences from incorrect ones and for constructing the logical structure of correct diagrams. A graph grammar is similar to a string grammar in the sense that the grammar consists of nite sets of labels for nodes and edges, an axiom, i.e., an initial graph, and a nite set of productions. Each node, respectively edge, can be equipped with an element of the label set. These elements can be used as types or as terminal or nonterminal symbols. Each production consists of a 3.3. DEFINING THE SYNTAX OF VISUAL LANGUAGES 121 left-hand-side graph (LHS) and a right-hand-side graph (RHS). It describes how an occurrence of the LHS in a graph can be replaced by the RHS; the resulting graph is said to be derived from the previous one by a rewrite step. The language of a graph grammar is the set of all graphs which can be derived in a nite sequence of rewrite steps from the initial graph. A hypergraph grammar is a graph grammar with hypergraphs instead of graphs and hyperedges instead of edges. The axiom of a hypergraph grammar is a graph consisting of a single hyperedge that visits each of the axioms nodes and that has a nonterminal label (\axiom label"). Similar to string grammars certain types of graph grammars with di erent generative power are distinguished. Each type is de ned by imposing certain restrictions on their productions and permitted derivation sequences. In Subsections 3.3.3 and 3.3.4 we will discuss three grammar types. The reason for restricting grammars is that the existence of parsers for a grammar and their e ciency highly depend on these restrictions. To motivate and illustrate the need for di erent graph grammar types we will use Nassi-Shneiderman diagrams (NSDs), Message Sequence Charts (MSCs), and a speci c kind of UML class diagrams as running examples. NSDs are a VL for representing structured programs [76] by attaching boxes to obtain rectangular diagrams. MSC is a language for the description of interaction between entities [57]. A diagram in MSC describes which messages are interchanged between process instances, and what internal actions they perform. UML [82] o ers a comprehensive VL for system modelling. It contains, e.g., class diagrams for describing object-oriented classes and their relationships. In our example we restrict class diagrams to classes represented by boxes and associations represented by arrows between classes. Associations may be augmented by association classes; this is visualized by a box representing the association class connected to the association with a dashed line. Figure 3.3 shows sample diagrams for NSD, MSC, and UML class diagrams. Also similar to string grammars, graph grammars can make use of attributes. As described in the following, we extend nodes and edges by attributes which may be used to represent actual positions and sizes of pictorial objects. E.g., for the NSD example, we will use two node attributes x and y for the coordinates of the box vertex represented by this node. Of course, other information like color and texture can also be represented by attributes. Among other things, the next subsection and Section 3.4 brie y discuss how attributed graph grammars may be used to de ne the syntax of VL sentences. Attributed graph grammars may process attributes quite similar as attributed string grammars [62]: each node and each (hyper)edge with the same label has to carry attributes of the 122 CHAPTER 3. APPLICATION TO VISUAL LANGUAGES input x while x>1 x := 3x-1 x := x/2 y n x even ? i1 i2 m0 m1 msc simple a works_for Job Tasks Employee Company (b) (c) (a) Figure 3.3: Instance of a Nassi-Shneiderman diagram (a), of a Message Sequence Chart (b), and of a UML class diagram (c). same types. Each production describes relationships between attributes which are used by the production. Whenever a production is going to be applied, i.e., when the LHS is matched with a host subgraph, matching attributes are identi ed. The relationships between the production's attributes either restrict application of the production (if the attributes of the matched subgraph do not satisfy required relations) or they contain information how to compute attribute values which were yet unknown. 3.3.3 Hypergraph Representation of Visual Sentences This subsection explains how hypergraphs and hypergraph grammars can be used to model visual sentences. Primarily, we will discuss graphs which are closely related to the visual appearance of a VL sentence, i.e., the sentence's SRG. Of course, the same graph concepts can be used for the ASG. However, the structure of ASGs is less determined by the sentence's visual appearance, but more by its logical structure and how the sentence is further processed within the VL environment. Since we focus on VLs as such but less on their use within VL environments, we emphasize SRGs. We will rst describe how hypergraphs may be used as a representation of visual sentences. We will then discuss how the syntax of certain VLs can be expressed by context-free hypergraph grammars. A more expressive grammar approach that allows to describe a less restricted syntax is discussed afterwards. Even more general kinds of hypergraph grammars may be used in this context, but we restrict our discussion to these kinds of syntax since they allow for more or less e cient parsing which is described in Section 3.5. Subsection 3.3.4 will continue with a di erent approach for representing visual sentences. Since parsing is not an issue of this approach, a more general kind of syntax is described there. 3.3. DEFINING THE SYNTAX OF VISUAL LANGUAGES 123 Hypergraph Model There are two ways to use graphs for modeling visual sentences: Pictorial objects are either represented by nodes and relationships between them by edges, or pictorial objects are represented by edges and nodes are used for connections between related objects. A variant of this rst approach is discussed in detail in Subsection 3.3.4. This subsection explains the second approach in detail. It is an appropriate means for representing visual sentences in a uniform way. In particular, free-hand editing as used by DiaGen (cf. Subsection 3.4.3) bene ts from this approach. However, it requires hypergraphs instead of graphs as in 

Application of graph transformation to visual languages

To increase speed and reliability of operation, multiple computers are replacing uniprocessors and wired-logic controllers in modern robots and industrial control systems. However, performance increases are not attained by such hardware alone. The operating software controlling the robots or control systems must exploit the possible parallelism of various control tasks in order to perform the necessary computations within given real-time and reliability constraints. Such software consists of both control programs written by application programmers and operating system software offering means of task scheduling, intertask communication, and device control.The Generalized Executive for real-time Multiprocessor applications (GEM) is an operating system that addresses several requirements of operating software. First, when using GEM, programmers can select one of two different types of tasks differing in size, called processes and microprocesses. Second, the scheduling calls offered by GEM permit the implementation of several models of task interaction. Third, GEM supports multiple models of communication with a parameterized communication mechanism. Fourth, GEM is closely coupled to prototype real-time programming environments that provide programming support for the models of computation offered by the operating system. GEM is being used on a multiprocessor with robotics application software of substantial size and complexity.

http://home.iitj.ac.in/~saurabh.heda/Papers/Research%20Papers/1987/High%20performance%20rtos%20for%20robotics.pdf

High-performance operating system primitives for robotics and real-time control systems

The increasing influence of visual technology on language environments that has resulted from the increasing availability of workstations is discussed. An evolution that began with the relatively straightforward translation of textual techniques into corresponding visual techniques and has progressed to uses of visual techniques that have no natural parallel using purely textural techniques is traced. Visual user interfaces, visual editing, and visual languages are surveyed. >

Influence of visual technology on the evolution of language environments

In the last few years, the rapid decline of computing costs, coupled with the sharp increase of personal computers and “canned” software, has expanded dramatically the population of the computer user community. More and more people today are using computers. However, to many people, the usefulness of a computer is bounded by the usefulness of the canned application software available for the computer. Application programs written for a mass audience seldom give every user all the capabilities that he/she needs. Those who wish to use the computer to do something beyond the capabilities of the canned programs discover that they have to “program.”

Visual Programming Languages: A Perspective and a Dimensional Analysis

This paper describes an implementation of a system for programming using structured charts with interactive graphical support. It provides a graphical editor for the user to interactively build and edit programs using Nassi-Shneiderman diagrams (NSD)1 as the structured control constructs of logic flow. It can interpret a program in NSD chart form, and the execution sequence of the NSD is displayed at a graphical terminal. On-line debugging and testing facilities are available which allow the user to examine and modify the program under execution. The system has been designed with the aim of supporting the development, debugging, testing, documentation and maintenance of programs in the same environment.

PIGS—A system for programming with interactive graphical support

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

N. Ng

Author Tools

Chat about Author

Papers

PIGS—A system for programming with interactive graphical support