Keyword-driven testing

Abstract: Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal. Various goals are possible; one involves rate of fault detection, a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during testing can provide faster feedback on the system under test and let software engineers begin correcting faults earlier than might otherwise be possible. One application of prioritization techniques involves regression testing, the retesting of software following modifications; in this context, prioritization techniques can take advantage of information gathered about the previous execution of test cases to obtain test case orderings. We describe several techniques for using test execution information to prioritize test cases for regression testing, including: 1) techniques that order test cases based on their total coverage of code components; 2) techniques that order test cases based on their coverage of code components not previously covered; and 3) techniques that order test cases based on their estimated ability to reveal faults in the code components that they cover. We report the results of several experiments in which we applied these techniques to various test suites for various programs and measured the rates of fault detection achieved by the prioritized test suites, comparing those rates to the rates achieved by untreated, randomly ordered, and optimally ordered suites.

Abstract: Publisher Summary This chapter discusses the different aspects of software testing techniques. A testing strategy may be based on one of two starting points and either the specification or the software is used as the basis for testing. Starting from the specification the required functions are identified. The software is then tested to assess whether they are provided. This is known as functional testing. It is found that if the strategy is based on deriving test data from the structure of a system this is known as structural testing. Functions which are included in the software, but not required, such as functions which relate to the access of data in a database but which are not specifically asked for by a user, are more likely to be identified by adopting a structural testing strategy in preference to a functional testing strategy. Functional testing has been termed a black box approach as it treats the program as a box with its contents hidden from view. Testers submit test cases to the program based on their understanding of the intended function of the program. An important component of functional testing is an oracle.

Abstract: This book gives a practical introduction to model-based testing, showing how to write models for testing purposes and how to use model-based testing tools to generate test suites. It is aimed at testers and software developers who wish to use model-based testing, rather than at tool-developers or academics. The book focuses on the mainstream practice of functional black-box testing and covers different styles of models, especially transition-based models (UML state machines) and pre/post models (UML/OCL specifications and B notation). The steps of applying model-based testing are demonstrated on examples and case studies from a variety of software domains, including embedded software and information systems. From this book you will learn: * The basic principles and terminology of model-based testing * How model-based testing differs from other testing processes * How model-based testing fits into typical software lifecycles such as agile methods and the Unified Process * The benefits and limitations of model-based testing, its cost effectiveness and how it can reduce time-to-market * A step-by-step process for applying model-based testing * How to write good models for model-based testing * How to use a variety of test selection criteria to control the tests that are generated from your models * How model-based testing can connect to existing automated test execution platforms such as Mercury Test Director, Java JUnit, and proprietary test execution environments * Presents the basic principles and terminology of model-based testing * Shows how model-based testing fits into the software lifecycle, its cost-effectiveness, and how it can reduce time to market * Offers guidance on how to use different kinds of modeling techniques, useful test generation strategies, how to apply model-based testing techniques to real applications using case studies

Abstract: Test case prioritization techniques schedule test cases for execution in an order that attempts to maximize some objective function. A variety of objective functions are applicable; one such function involves rate of fault detection-a measure of how quickly faults are detected within the testing process. An improved rate of fault detection during regression testing can provide faster feedback on a system under regression test and let debuggers begin their work earlier than might otherwise be possible. In this paper we describe several techniques for prioritizing test cases and report our empirical results measuring the effectiveness of these techniques for improving rate of fault detection. The results provide insights into the tradeoffs among various techniques for test case prioritization.