Journal Article10.1002/STVR.1572
Coverage-based regression test case selection, minimization and prioritization: a case study on an industrial system
112
TL;DR: This paper presents a case study of coverage‐based regression testing techniques on a real world industrial system with real regression faults, and shows that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates.
read more
Abstract: This paper presents a case study of coverage-based regression testing techniques on a real world industrial system with real regression faults. The study evaluates four common prioritization techniques, a test selection technique, a test suite minimization technique and a hybrid approach that combines selection and minimization. The study also examines the effects of using various coverage criteria on the effectiveness of the studied approaches. The results show that prioritization techniques that are based on additional coverage with finer grained coverage criteria perform significantly better in fault detection rates. The study also reveals that using modification information in prioritization techniques does not significantly enhance fault detection rates. The results show that test selection does not provide significant savings in execution cost <2%, which might be attributed to the nature of the changes made to the system. Test suite minimization using finer grained coverage criteria could provide significant savings in execution cost 79.5% while maintaining a fault detection capability level above 70%, thus representing a possible trade-off. The hybrid technique did not provide a significant improvement over traditional minimization techniques. Copyright © 2015John Wiley & Sons, Ltd.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Reinforcement learning for automatic test case prioritization and selection in continuous integration
Helge Spieker,Arnaud Gotlieb,Dusica Marijan,Morten Mossige +3 more
- 10 Jul 2017
TL;DR: In this article, the Retecs method uses reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure history, in a constantly changing environment.
248
Test case prioritization approaches in regression testing: A systematic literature review
TL;DR: This review examines and classify the current test case prioritization approaches in TCP based on the articulated research questions and found that variations in the starting point of TCP process among the approaches provide a different timeline and benefit to project manager to choose which approaches suite with the project schedule and available resources.
233
Hybrid regression test selection
Lingming Zhang
- 27 May 2018
TL;DR: The experimental results on 2707 revisions of 32 projects, totalling over 124 Million LoC, demonstrate that HyRTS outperforms state-of-the-art FRTS significantly in terms of selected test ratio and the offline testing time.
132
Effective Regression Test Case Selection: A Systematic Literature Review
TL;DR: This systematic literature review presents state-of-the-art research in effective regression test case selection techniques and observed that 70% of the studies being analyzed used cost as the effectiveness measure compared to 31% that use fault-detection capability and 16% that used coverage.
108
Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration
TL;DR: This paper introduces Retecs, a new method for automatically learning test case selection and prioritization in CI with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases.
69
References
•Book
Experimentation in Software Engineering: An Introduction
Claes Wohlin,Per Runeson,Martin Höst,Magnus C. Ohlsson,Bjöorn Regnell,Anders Wesslén +5 more
- 27 Sep 2011
TL;DR: The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering, and to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering.
3.1K
Regression testing minimization, selection and prioritization: a survey
Shin Yoo,Mark Harman +1 more
TL;DR: This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
1.5K
Prioritizing test cases for regression testing
TL;DR: Test case prioritization techniques schedule test cases for execution in an order that attempts to increase their effectiveness at meeting some performance goal as discussed by the authors, such as rate of fault detection, a measure of how quickly faults are detected within the testing process.
1.3K
Prioritizing test cases for regression testing
Sebastian Elbaum,Alexey G. Malishevsky,Gregg Rothermel +2 more
- 01 Aug 2000
TL;DR: Can prioritization techniques be effective when aimed at specific modified versions; what tradeoffs exist between fine granularity and coarse granularity prioritized techniques; and can the incorporation of measures of fault proneness into prioritization technique improve their effectiveness?
A practical guide for using statistical tests to assess randomized algorithms in software engineering
Andrea Arcuri,Lionel C. Briand +1 more
- 21 May 2011
TL;DR: It is shown that randomized algorithms are used in a significant percentage of papers but that, in most cases, randomness is not properly accounted for, which casts doubts on the validity of most empirical results assessing randomized algorithms.
1K