Abstract: Preface. Dedication. Chapter Summaries. I. INTRODUCTION. 1. A Fault Model to Guide Software Testing. The Purpose of Software Testing. Understanding Software Behavior. Understanding Software's Environment. The Human User. File System User. The Operating System User. The Software User. Understanding Software's Capabilities. Testing Input. Testing Output. Testing Data. Testing Competition. Summary and Conclusion. Exercises. References. II. USER INTERFACE ATTACKS. 2. Testing from the User Interface: Inputs and Outputs. Using the Fault Model to Guide Testing. Exploring the Input Domain. First Attack: Apply inputs that force all the error messages to occur. Second Attack: Apply inputs that force the software to establish default values. Third Attack: Explore allowable character sets and data types. Fourth Attack: Overflow input buffers. Fifth Attack: Find inputs that may interact and test various combinations of their values. Sixth Attack: Repeat the same input or series of inputs numerous times. Exploring Outputs. Seventh Attack: Force different outputs to be generated for each input. Eighth Attack: Force invalid outputs to be generated. Ninth Attack: Force properties of an output to change. Tenth Attack: Force the screen to refresh. Conclusion. Exercises. References. 3. Testing from the User Interface: Data and Computation. Testing Inside the Box. Exploring Stored Data. Eleventh Attack: Apply inputs using a variety of initial conditions. Twelfth Attack: Force a data structure to store too many/too few values. Thirteenth Attack: Investigate alternate ways to modify internal data constraints. Exploring Computation and Feature Interaction. Fifteenth Attack: Force a function to call itself recursively. Sixteenth Attack: Force computation results to be too large or too small. Seventeenth Attack: Find features that share data or interact poorly. Conclusion. Exercises. III. SYSTEM INTERFACE ATTACKS. 4. Testing from the File System Interface. Attacking Software from the File System Interface. Media-based Attacks. First Attack: Inject faults that simulate memory access problems. Second Attack: Inject faults that simulate network problems. Third Attack: Damage the media. File-based Attacks. Fourth Attack: Assign an invalid file name. Fifth Attack: Vary file access permissions. Sixth Attack: Vary/corrupt file contents. Exercises. 5. Testing from the Software/OS Interface. Attacking Software from Software Interfaces. Record-and-Simulate Attacks. Observe-and-Fail Attacks. Conclusion. Exercises. IV. CONCLUSION. 6. Some Parting Advice. You'll Never Know Everything. Bug Hunts. Friday Afternoon Bug Fests. Conclusion. References. APPENDICES. Annotated Glossary of Programming Terms. Appendix A. Testing Exception and Error Cases Using Runtime Fault Injection. Introduction. A Mechanism for Runtime Fault Injection. Fault Selection. Conclusions. Acknowledgments. References. Appendix B. Using HEAT: The Hostile Environment Application Tester. Canned HEAT User Guide. The Application Band. The Monitor Band. Fault Injection Bands and Their Functionality. The Network Band. Disk Storage. Memory. Appendix C. What is Software Testing? And Why is it so Hard? Introduction. The Software Testing Process. Phase One: Modeling the Software's Environment. Phase Two: Selecting Test Scenarios. Phase Three: Running and Evaluating Test Scenarios. Phase Four: Measuring Testing Progress. Conclusion. References. The Software Testing Problem.

Abstract: Testing in order to produce software of high reliability is an area of major concern in software engineering. In an effort to find efficient methods of testing, the comparison of partition and random sampling testing methods has received considerable attention in the literature. A standard criterion for comparisons between random and partition testing, based on their expected efficacy in program debugging, is the probability of detecting at least one failure causing input in the program's domain. However, the goal in software testing is usually to find as many faults as possible in a reasonable period of time, and therefore stochastic comparisons of the number of faults obtained in partition and random testing may provide more valuable information on which testing procedures to use. We establish various conditions which guarantee that the number of faults discovered in partition testing is stochastically greater than the number discovered in random testing (using a fixed total sample size) for many of the well-established stochastic orders (including the usual stochastic order, the hazard rate order, the likelihood ratio order, and the variability order). The results established also allow us to obtain both upper and lower bounds with these stochastic orders for the sum of n independent Bernoulli random trials (with varying probability of success) in terms of the binomial distribution with parameters n and p.

Abstract: The purpose of the article is to demonstrate an equivalence testing software application written under SAS Institute software designed for use by pharmaceutical and other medical research professionals. Besides making the entire equivalence testing procedure easier and more efficient, the application "EquivEasy" offers three main advantages over similar software: (a) testing for 3/spl times/3 in addition to 2/spl times/2 crossover designs, (b) familiar SAS user environment, and (c) export flexibility (MS Word, PDF, HTML). Two case studies are presented with report results provided in tabular and graphical form.

Abstract: Random testing has been used for years in both software and hardware testing. It is well known that in random testing each test requires to be selected randomly regardless of the tests previously generated. However, random testing could be inefficient for its random selection of test patterns. This paper, based on random testing, introduces the concept of Maximum Distance Testing (MDT) for VLSI circuits in which the total distance among all test patterns is chosen maximal so that the set of faults detected by one test pattern is as different as possible from that of faults detected by the tests previously applied. The procedure for constructing a Maximum Distance Testing Sequence (MDTS) is described in detail. Experimental results on Benchmark as well as other circuits are also given to evaluate the performances of our new approach.

Abstract: As the complexity of applications and therefore of their testing process grows, the importance of automating the testing activity increases. The testing process includes test case generation, test sequencing, oracle construction, test execution and result interpretation. Automatic generation of test cases from formal specifications has received considerable attention. Relatively little work has been reported, however, on constructing oracles for supporting efficient and automatic execution of such test cases. We present a technique for constructing a CORBA-supported VDM oracle for black-box testing starting from a VDM-SL specification. This specification is used to automatically verify the results of operations implemented in a high-level programming language. We present a case study of the technique applied to a Java application for generic access control. The technique is applicable to any CORBA-compliant programming language.

Abstract: Analyzing defect behavior is becoming increasingly difficult with the rising significance of defects that depend on random parameters. Such unpredictable parameters can affect various types of test escape. The concept of detection domains can help sort out the behavior of these test escapes.

Abstract: Most techniques proposed for regression testing are either black- or white-box techniques, i.e. they solely consider either the specification or implementation of a program. However a combined technique can lead to significant savings in testing costs as some testing tasks need to be carried out once. Furthermore, the same test cases can cover both specification and implementation, reducing the total number of test cases. This paper shows the application of class specification implementation graphs (CSIGs) in regression testing. The distinguishing feature of CSIGs front existing representations is that each method of a class is shown from two perspectives, namely the specification and implementation view. Moreover a test suite reduction strategy has been incorporated to the CSIG construction algorithm to decrease the total number of test cases required. In this paper we show how CSIGs can be combined with an existing test case selection algorithm for regression testing.

Abstract: Shows that current techniques that use BISTs for testing CUTs often make it impossible to distinguish which one is faulty: a CUT or a MISR. The paper shows a number of additional benefits following from making use of BIST to test chips, FPGA circuits etc., if the only effect were that the testing of an MISR would confirm credibly the correctness of the MISR. Furthermore, the paper proposes such modification of the MISR compactor structure that it makes possible to obtain reliable results of testing. Additionally, an effective technique of testing such compactor is presented and a minimal number of test clock cycles that is required for full testing its correctness is determined.

Abstract: Recently, it has become popular to use applications on window operating systems, such as Microsoft Windows and GNOME, and for this we attempt to use the GUI specific testing approach. Legacy type of approaches, such as control flow path and data flow path testing, are not always effective testing methods for GUI application. Thus, model-based testing was newly established and often used in GUI application testing. But the model-based testing method is not yet mature and has not been researched enough. Our developed model-based testing focuses not only on application behavior but also on data and external factors. In addition, we realized that model-based testing lacks the ability to find some types of defect. In this paper, we will clarify which factors are lacking and offer solutions with case studies.

Abstract: To enhance the reliability and quality, software systems should be tested. The object-oriented paradigm is regarded as a most promising methodology for software development. The testing of object-oriented software systems is more difficult and more complex than that for traditional programming systems. We describe an approach for object-oriented testing at the method-level. Our approach applies the data flow analysis method combined with the state transition diagram technique, access protection checking technique, and compiler technique to the object-oriented method-level tests. Based on this approach, a prototype for C++ programs has been implemented. Some experimental results on this prototype are presented. The experimental results on the prototype indicate that the approach can reveal data flow errors in C++ programs as anticipation. It supplements our previous approaches for object-oriented testing at the class and cluster-levels.

Abstract: Generating test data sets which are sufficiently large to effectively cover all the tests required before a software component can be certified as reliable is a time consuming and error-prone task if carried out manually. A key parameter when testing collections is the size of the collection to be tested: an automatic test generator builds a set of collections containing n elements where n ranges from 0 to n/sub crit/. Data coverage analysis allows us to determine rigorously a collection size such that testing with collections of size > n/sub crit/ does not provide any further useful information, i.e. will not uncover any new faults. We conducted a series of experiments on modules from the C++ Standard Template Library which were seeded with errors. Using a test model appropriate to each module, we generated data sets of sizes up to and exceeding the predicted value of n/sub crit/ and verified that after all collections of size /spl les/n/sub crit/ have been tested, no further errors are discovered. Data coverage was also compared with statement coverage testing and random test data set generation. The three testing techniques were compared for effectiveness at revealing errors compared to the number of test data sets used. Statement coverage testing was confirmed as the cheapest, in the sense that it produces its maximal effect for the smallest number of tests applied, but the least effective technique in terms of numbers of errors uncovered. Data coverage was significantly better than random test generation: it uncovered more faults with fewer tests at every point.

Abstract: The authors compare the partition testing and the random testing under different assumption (Certainty and Uncertainty models). It is concluded that under certain failure rate assumption, the random testing is superior to the partition testing and under uncertainty assumption, the partition testing is superior to the random testing. These results are of practical interest to software testing.

Abstract: To analyze the errors of processing data, the testing principle for jet elements is introduced and the property of testing system is theoretically and experimentally studied. On the basis of the above, the method of processing data is presented and the error formulae, which are the functions of the testing system property, are derived. Finally, the methods of reducing the errors are provided. The measured results are in correspondence with the theoretical conclusion.

Abstract: Genetic Programming (GP) has recently-been applied to solve problems in several areas. It has the goal of inducing programs from test cases by using the concepts of Darwin's evolution theory. On the other hand, software testing, that is a fundamental and expensive activity for software quality assurance, has the objective of generating test cases from the program being tested. In this sense, a symmetry between induction of programs based on GP and testing is noticed. Based on such symmetry, this work presents GPTesT, a testing tool based on GP. Fault-based testing criteria generally derive test data using a set of mutant operators to produce alternatives that differ from the program under testing by a simple modification. GPtesT uses a set of alternatives genetically derived, which allow the test of interactions between faults. GPTesT implements two test procedures respectively for guiding the selection and evaluation of test data sets. Examples with these procedures show that the approach can be used as a testing criterion.