Advances in Model-Based Software Testing 

Abstract: Software testing is a key procedure to ensure high quality and reliability of software programs. The key issue in software testing is the selection and evaluation of different test cases. Code coverage has been proposed to be an estimator for testing effectiveness, but it remains a controversial topic which lacks of support from empirical data. In this study, we hypothesize that the estimation of code coverage on testing effectiveness varies under different testing profiles. To evaluate the performance of code coverage, we employ coverage testing and mutation testing in our experiment to investigate the relationship between code coverage and fault detection capability under different testing profiles. From our experimental data, code coverage is simply a moderate indicator for the capability of fault detection on the whole test set. However, it is clearly a good estimator for the fault detection of exceptional test cases, but a poor one for test cases in normal operations. For other testing profiles, such as functional testing and random testing, the correlation between code coverage and fault coverage is higher in functional test than in random testing, although these different testing profiles are complementary in the whole test set. The effects of different coverage metrics are also addressed in our experiment.

Abstract: During regression testing, a modified system needs to be retested using the existing test suite. Since test suites may be very large, developers are interested in detecting faults in the system as early as possible. Test prioritization orders test cases for execution to increase potentially the chances of early fault detection during retesting. Most of the existing test prioritization methods are based on the code of the system, but model-based test prioritization has been recently proposed. System modeling is a widely used technique to model state-based systems. System models may not only be used to generate test cases but also to prioritize tests. In model-based prioritization, information collected during execution of a model is used to prioritize tests for execution. In this paper we present several model-based test prioritization heuristics. The major motivation to develop these heuristics was simplicity and effectiveness in early fault detection. We have conducted a small experimental study in which we experimentally compared model-based test prioritization heuristics. The results have shown that some simple heuristic methods can be as effective in early fault detection as more complex ones.

Abstract: Model-based regression testing is an important activity that ensures the reliability of evolving software. One of the major issues in this type of testing is the optimal selection of test-cases to test the affected portion of the software. In this paper, we present a UML based selective regression testing strategy that uses state machines and class diagrams for change identification. We identify the changes using the UML 2.1 semantics of state machines and class diagram. The changes are classified as Class-driven (obtained from class diagram) and State-driven (obtained from state machine). The Class-driven changes are important as these changes are not reflected on the state machines and they might be helpful in identifying some fault-revealing test cases. With the help of the identified changes, we classify the test cases of the test suite as Obsolete, Reusable, and Retestable. We apply the approach on a case study to demonstrate its validity.

Abstract: A regression test suite (RTS) is constructed to ensure that the changed parts of the system under test (SUT) behave as desired and that the unchanged parts of the SUT are not adversely affected. Model-based testing is a system testing technique in which systems are modeled by formal description languages, e.g., Extended Finite State Machine (EFSM) models. In this paper, a model-based RTS generation method based on EFSM dependence analysis is proposed. Twelve types of dependences are identified related to three types of elementary modifications (EMs), i.e., adding, deleting, and changing transitions in an EFSM model representing an SUT. These dependences capture the effects of the model on the EMs, the effects of the EMs on the model, and the side-effects of the EMs. The proposed method constructs an RTS by covering all occurrences of these dependences caused in a given EFSM model by a given set of EMs.