Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.

Regression testing minimization, selection and prioritization: a survey

The present invention relates to a testing tool for testing of complex software systems having a plurality of interrelated system components and a suite of test scenarios for testing the performance of the plurality of interrelated system components. The testing tool includes an Automated Multidimensional Traceability Matrix system for determining linkages between interrelated system components, a means for identifying a change in one or more of the interrelated system components, a means for applying the Automated Multidimensional Traceability Matrix, a means for executing all of or a subset of the test scenarios associated with the interrelated system components that may be affected by the change and a means for evaluating the results of the executed test scenarios.

Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems

Regression testing is verifying that previously functioning software remains after a change. With the goal of finding a basis for further research in a joint industry-academia research project, we conducted a systematic review of empirical evaluations of regression test selection techniques. We identified 27 papers reporting 36 empirical studies, 21 experiments and 15 case studies. In total 28 techniques for regression test selection are evaluated. We present a qualitative analysis of the findings, an overview of techniques for regression test selection and related empirical evidence. No technique was found clearly superior since the results depend on many varying factors. We identified a need for empirical studies where concepts are evaluated rather than small variations in technical implementations.

/pdf/a-systematic-review-on-regression-test-selection-techniques-18fawa0xny.pdf

A systematic review on regression test selection techniques

The increase in size and complexity of modern software systems requires scalable, systematic, and automated testing approaches. Model-based testing (MBT), as a systematic and automated test case generation technique, is being successfully applied to verify industrial-scale systems and is supported by commercial tools. However, scalability is still an open issue for large systems, as in practice there are limits to the amount of testing that can be performed in industrial contexts. Even with standard coverage criteria, the resulting test suites generated by MBT techniques can be very large and expensive to execute, especially for system level testing on real deployment platforms and network facilities. Therefore, a scalable MBT technique should be flexible regarding the size of the generated test suites and should be easily accommodated to fit resource and time constraints. Our approach is to select a subset of the generated test suite in such a way that it can be realistically executed and analyzed within the time and resource constraints, while preserving the fault revealing power of the original test suite to a maximum extent. In this article, to address this problem, we introduce a family of similarity-based test case selection techniques for test suites generated from state machines. We evaluate 320 different similarity-based selection techniques and then compare the effectiveness of the best similarity-based selection technique with other common selection techniques in the literature. The results based on two industrial case studies, in the domain of embedded systems, show significant benefits and a large improvement in performance when using a similarity-based approach. We complement these analyses with further studies on the scalability of the technique and the effects of failure rate on its effectiveness. We also propose a method to identify optimal tradeoffs between the number of test cases to run and fault detection.

Achieving scalable model-based testing through test case diversity

Test case prioritization techniques have been shown to be beneficial for improving regression-testing activities. With prioritization, the rate of fault detection is improved, thus allowing testers to detect faults earlier in the system-testing phase. Most of the prioritization techniques to date have been code coverage-based. These techniques may treat all faults equally. We build upon prior test case prioritization research with two main goals: (1) to improve user-perceived software quality in a cost effective way by considering potential defect severity and (2) to improve the rate of detection of severe faults during system-level testing of new code and regression testing of existing code. We present a value-driven approach to system-level test case prioritization called the prioritization of requirements for test (PORT). PORT prioritizes system test cases based upon four factors: requirements volatility, customer priority, implementation complexity, and fault proneness of the requirements. We conducted a PORT case study on four projects developed by students in advanced graduate software testing class. Our results show that PORT prioritization at the system level improves the rate of detection of severe faults. Additionally, customer priority was shown to be one of the most important prioritization factors contributing to the improved rate of fault detection.

/pdf/system-test-case-prioritization-of-new-and-regression-test-32bz35opkg.pdf

System test case prioritization of new and regression test cases

A regression test suite (RTS) is constructed to ensure that the changed parts of the system under test (SUT) behave as desired and that the unchanged parts of the SUT are not adversely affected. Model-based testing is a system testing technique in which systems are modeled by formal description languages, e.g., Extended Finite State Machine (EFSM) models. In this paper, a model-based RTS generation method based on EFSM dependence analysis is proposed. Twelve types of dependences are identified related to three types of elementary modifications (EMs), i.e., adding, deleting, and changing transitions in an EFSM model representing an SUT. These dependences capture the effects of the model on the EMs, the effects of the EMs on the model, and the side-effects of the EMs. The proposed method constructs an RTS by covering all occurrences of these dependences caused in a given EFSM model by a given set of EMs.

Model-based regression test suite generation using dependence analysis

A model-based regression test suite (RTS) reduction method based on Extended Finite State Machine (EFSM) dependence analysis is proposed. Given an EFSM representing the requirements of a system under test (SUT) and a set of elementary modifications (EMs) on the EFSM, interaction patterns are identified related to each type of EMs, i.e., adding, deleting, and changing transitions in the EFSM. These interaction patterns capture the effects of the model on the EMs, the effects of the EMs on the model, and the side-effects of the EMs. The proposed method reduces the size of a given RTS by examining interaction patterns covered by each test case in the given RTS.

Regression test suite reduction using extended dependence analysis

How to measure a test process is a required competence for an effective software test manager. Effective test process measurement is essential for designing, evaluating and managing a cost-effective test strategy. Although some existing research has proposed many metrics for these purposes, most of those metrics are ignored or used individually. This paper summarizes a test process measurement pilot project of the IBM Electronic Commerce Development (ECD) test teams, and analyzes the effectiveness of using a "metric set" of complementary metrics for cost, time, and quality to measure the quality of test process based on the results of our project. Details are given for a "fictitious" project in order to explain the detailed considerations that go into adopting a cost-effective metric strategy. Further discussions about implementation problems, test wisdom learned in the project, and cost-effectiveness of the measurement activities and metrics are also presented. We conclude with suggestions for extensions of the work into the area of real-time, in-phase, in-process metrics.

Effective test metrics for test strategy evolution

This paper proposes a model-based regression test suite reduction method. The proposed method considers an SDL model representing the requirements of a system under test and a set of modifications on this model, applies dependence analysis to identify interaction patterns related to each type of modifications, i.e., adding, deleting, and changing transitions in the SDL model, and reduces the size of a given regression test suite by examining interaction patterns covered by each test case in the test suite. Results of empirical studies are reported. Copyright © 2009 John Wiley & Sons, Ltd.

Regression test suite reduction based on SDL models of system requirements

In this paper, we describe a formal, model-driven, CORBA-based approach to developing and testing e-commerce systems. We indicate advantages and limitations of formal verification techniques using the Specification and Description Language (SDL), and relate the CORBA-based distributed object architecture to standard test methods and TTCN, the international standard test language. Finally, we enumerate industrial challenges and best practices at one of the electronic commerce software test organizations in IBM, and suggest a strategy for adapting formal test methods to the evolving industrial e-commerce test process.

Formal verification and validation for e-commerce: theory and best practices

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Yanping Chen

Author Tools

Chat about Author