Abstract: Most testing methods generate test descriptions which define the desired characteristics of the input data in a test case. This paper describes the use of accumulated knowledge about a problem domain to refine these test descriptions, with the goal of increasing the probability that the input data generated from the refined test descriptions will reveal faults in a software system. A knowledge base is introduced to hold information about object semantics and object class/subclass relationships. Knowledge accumulates with experience in a particular domain and can be focused on those objects and relationships in that domain which experience has shown to be error‐prone. This paper also defines a knowledge‐driven functional testing (KDFT) method which derives test descriptions from a formal specification and refines these descriptions using that knowledge base. A case study of the KDFT method using data from a previous study of the launch intercept control problem is described. These preliminary results indicate that knowledge‐based refinement of test descriptions can dramatically improve their ability to detect certain classes of faults.

Abstract: Software quality is the elusive goal of the IT industry. Many authors of both software engineering books and tools have led us to believe that software quality is the inevitable by-product of adherence to their particular approach. This is the 'do what I say and everything will be OK school of software engineering thought. The result has been that too many people have failed to recognise the importance of modelling and measuring software quality as an independent activity. Consequently this book is a most welcome addition to the software literature. It is an excellent and timely book, being probably the first devoted exclusively to the general issue of software quality. The author takes an eclectic view of quality; he does not force one particular model or method on us, and this is a major strength of the book. The author is concerned with the fundamental problems of defining and measuring software quality. The fact that he is able to relate this to the practical problems of management makes it all the more significant. It was the author's intention to bridge the gap between software engineering and software management, and he has achieved this very well. The book is divided into three parts. In part one (theory of software quality) there is an excellent introductory chapter on different views of quality followed by four chapters concerned with various models and measures of software quality. These are impressive because the author manages to make accessible much work which is very current, including recent work in Japan and on European Community ESPRIT projects. Part two is concerned with the management of software quality. This contains: a chapter which provides a good overview of software engineering methodologies and CASE tools; a chapter on quality management systems, including the concepts of total quality management (TQM) and statistical quality control, and an excellent summary of relevant standards; a chapter devoted to case studies; and a chapter which looks at the likely impact of CASE tools, TQM and standards. The final part of the book is actually just a single short chapter looking beyond software quality to IT effectiveness. Much of the material in the book, including the many lively figures, appears to have come from the author's MSc teaching material. As a result, the book is easy to read, the content looks good, and it would be readily usable as a course text (although there are no exercises). The author suggests it is appropriate for MSc courses and possibly final year undergraduate programmes. I believe it could be used on a second year course; moreover it would be very good background reading (starting from the first year) for any students doing degrees in software engineering. The book would also be a valuable asset to IT professionals.

Abstract: Cleanroom programming and code inspections independently provide evidence that it is more efficient to postpone the testing of code to a later stage than is usually done. This paper argues that an additional gain in quality and efficiency of development can be obtained by structuring inspections by means of an inspection protocol. The written part of such a protocol is prepared by the programmer before the inspection. It is modelled on Floyd's method for the verification of flowcharts. However, the protocol differs from Floyd's method in being applicable in practice. Structured inspections gain this advantage by not attempting to be a proof; they are no more than an articulation of existing forms of inspection. With the usual method of structured programming it may be difficult to prepare the inspection protol. On the other hand, ‘assertion-driven programming’ (of which an example is included in this paper) not only facilitates protocol preparation, but also the coding itself.

Abstract: The testing of modular software systems can be divided into a module testing phase and an integration testing phase. While module testing checks the modules separately, integration testing examines the use of interfaces in a modular system. Integration testing allows errors to be found which cannot be found by module testing. The aim of this paper is to propose a new approach to integration testing. The main principle is to transfer and adapt module testing methods to the level of integration testing. The approach is described for control flow and data flow oriented testing methods. To decrease the testing effort and increase the probability of finding errors, integration testing can be limited to statically detectable anomalous applications of interfaces. This is accomplished by the combination of static analysis with dynamic execution and by the possibility of using information already provided by the module tests. To find further test data to execute interfaces, symbolic execution is applied. One great advantage here is to prove whether statically determined interface anomalies can be dynamically executed and can therefore occur at all.

Abstract: What does it mean to say that an item of software has been tested? Unfortunately, currently accepted standards are inadequate to give the confidence the user needs and the meaningful objective for the supplier. This paper assesses the currently available standards, mainly in the component testing area and advocates that the British Computer Society proto-standard should be taken as a basis for a formal standard. The paper is intended for those concerned with software quality.

Abstract: A collection of 69 references drawn from books, papers and conference proceedings on the subject of software testing is presented. Each reference is accompanied by a paragraph describing its contents. The aim when selecting the references was to minimize the number and maximize the coverage of the subject. For this reason, the bibliography is primarily intended for the researcher or practitioner who is relatively new to the subject. The bibliography is organized according to the following sections: general introductions; background; testing methods; experimental studies; and tools.

Abstract: This book is intended to show both the manager and the industrial practitioner the power of statistical methods for producing more competitive products with a minimal effort. Its aim is to serve as an ‘easy-to-use’ manual for professionals with minimal statistical training. As a general comment, most descriptions of the methods are too sketchy, giving little explanation and no theoretical background at all. I cannot quite agree with the author that theory may only confuse the practitioner. There are many good examples included, but they cannot really make up for the lack of explanations in the general part. Each chapter starts with an overview discussion. A ‘getting started’ chapter helps the reader who does not want to study the whole book to find the chapters relevant to his or her problem. Most chapters end with some ‘do-itsmarter’ considerations. These sections summarize the material covered and give useful advice and cross-references to other chapters. There are 22 chapters in all, covering a wide range of topics including factorial experiments, reliability testing and statistical process control. The necessity of applying a combination of these techniques is strongly emphasized. Some mathematics, computer programs, computer output descriptions and various reference tables are collected in the appendices. Chapter 4, ‘Descriptive statistics and experimental traps’ covers much more than is indicated by its title; all basic concepts such as sample and population, hypothesis testing, confidence intervals, random sampling, experimental error, etc., are presented within 20 pages. Of course, there cannot be much space left for explanations! For example, it is frequently mentioned that samples have to be taken at random, leaving it to the reader to create or maintain his or her private idea of randomness. Equally, the concept of a hypothesis test remains vague. There is no word about possible misinterpretations or the imbalance between the null hypothesis and the alternative. Chapters 7 and 8 give formulae for tests and confidence intervals for means, standard deviations and proportions. All testing problems are accompanied by sample size considerations. Non-parametric tests are not included. The t-test is recommended even for obviously non-normal data. This is surprising, as the author repeatedly emphasizes the need for quick methods requiring minimal sample sizes. Chapters 10 to 19 are devoted to the main topics of the book and a wide range of material is included. This part gives a good overview of the techniques used in industrial applications, but again, it is too brief to serve as a helpful guide for the novice. In conclusion, the book describes many statistical methods which should certainly be of interest for everybody involved in software development or quality assurance and control. However, the non-statistician who wants to apply these methods will have to seek additional help; the statistician can choose among excellent books on factorial experiments, reliability theory or process control; and the manager who needs to be economical might be reluctant to spend more than f50 or too busy to read 516 pages merely to get an overview.

Abstract: The B-Method is a complete formal development process for mathematically transforming software systems from specification through to code. This article provides the reader with an overview of the process including a description of the language used for specifying systems (Abstract Machine Notation) and demonstrates its application by a simple, real-life case study. The method has tool support in the form of a tool-kit which is described here and applied to the case study. The results of the case study show how a system can be validated and verified in the early stages of its development through proof of the mathematical specification and an animating tool.

Abstract: The verification of an SR flip‐flop is contrasted with a testing approach.

Abstract: A collection of 69 references drawn from books, papers and conference proceedings on the subject of software testing is presented. Each reference is accompanied by a paragraph describing its contents. The aim when selecting the references was to minimize the number and maximize the coverage of the subject. For this reason, the bibliography is primarily intended for the researcher or practitioner who is relatively new to the subject. The bibliography is organized according to the following sections: general introductions; background; testing methods; experimental studies; and tools.

Abstract: ‘Propagation, infection, and execution analysis’ (termed PIE) is used for predicting where faults can more easily hide in software. To make such predictions, programs are dynamically executed with test cases, and information concerning the test cases is collected into a histogram, each bin of which represents a single test case. The score in a bin predicts the likelihood that the test case will reveal a fault through the production of a failure (if a fault exists in the set of program locations that the test case executes). Preliminary experiments using program mutations suggest that the histogram technique presented in this paper can rank test cases according to their fault revealing ability.

Abstract: Z is a formal notation for writing system specifications that has been growing in popularity over recent years. This paper examines some of the issues involved in applying a ‘partition based’ testing method to a system specified in Z. Details of an extensive case study are given, from specification and implementation of the system to the development and execution of test cases. The strategy is found to have benefits compared to those based on less formal specifications, but there are limitations to the approach, and difficulties that need addressing.

Abstract: The term ‘inductive inference’ denotes the process of hypothesizing a general rule from examples. It can be considered as the inverse process of program testing, which is a process of sampling the behaviour of a program and gathering confidence in the quality of the software from the samples. As one of the fundamental and ubiquitous components of intelligent behaviour, much effort has been spent on both the theory and practice of inductive inference as a branch of artificial intelligence. In this paper, software testing and inductive inference are reviewed to illustrate how the rich and solid theory of inductive inference can be used to study the foundations of software testing.