Journal Article10.1145/2768829
Do Automatically Generated Test Cases Make Debugging Easier? An Experimental Assessment of Debugging Effectiveness and Efficiency
54
TL;DR: It is shown that automatically generated test cases are as useful for debugging as manual test cases and, for less experienced developers, automatic tests are more useful on average due to their lower static and dynamic complexity.
read more
Abstract: Several techniques and tools have been proposed for the automatic generation of test cases. Usually, these tools are evaluated in terms of fault-revealing or coverage capability, but their impact on the manual debugging activity is not considered. The question is whether automatically generated test cases are equally effective in supporting debugging as manually written tests.We conducted a family of three experiments (five replications) with humans (in total, 55 subjects) to assess whether the features of automatically generated test cases, which make them less readable and understandable (e.g., unclear test scenarios, meaningless identifiers), have an impact on the effectiveness and efficiency of debugging. The first two experiments compare different test case generation tools (Randoop vs. EvoSuite). The third experiment investigates the role of code identifiers in test cases (obfuscated vs. original identifiers), since a major difference between manual and automatically generated test cases is that the latter contain meaningless (obfuscated) identifiers.We show that automatically generated test cases are as useful for debugging as manual test cases. Furthermore, we find that, for less experienced developers, automatic tests are more useful on average due to their lower static and dynamic complexity.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Automatic generation of oracles for exceptional behaviors
Alberto Goffi,Alessandra Gorla,Michael D. Ernst,Mauro Pezzè +3 more
- 18 Jul 2016
TL;DR: Toradocu is proposed, a technique that automatically creates test oracles for exceptional behaviors from Javadoc comments that uses a combination of natural language processing and run-time instrumentation.
118
The impact of test case summaries on bug fixing performance: an empirical investigation
Sebastiano Panichella,Annibale Panichella,Moritz Beller,Andy Zaidman,Harald C. Gall +4 more
- 14 May 2016
TL;DR: An approach which automatically generates test case summaries of the portion of code exercised by each individual test, thereby improving understandability, is proposed, which can complement the current techniques around automated unit test generation or search-based techniques designed to generate a possibly minimal set of test cases.
Human Competitiveness of Genetic Programming in Spectrum-Based Fault Localisation: Theoretical and Empirical Analysis
TL;DR: This work reports on the application of Genetic Programming to Software Fault Localisation, a problem in the area of Search-Based Software Engineering (SBSE), and proves that no future human investigation could outperform the evolved solutions.
79
Unit Test Generation During Software Development: EvoSuite Plugins for Maven, IntelliJ and Jenkins
Andrea Arcuri,Jose Campos,Gordon Fraser +2 more
- 11 Apr 2016
TL;DR: The resulting architecture of the plugins, and the challenges arising when building such plugins, are discussed, which are targeted for the EvoSuite tool and can be adapted and reused for other test generation tools as well.
References
A Simple Sequentially Rejective Multiple Test Procedure
TL;DR: In this paper, a simple and widely accepted multiple test procedure of the sequentially rejective type is presented, i.e. hypotheses are rejected one at a time until no further rejections can be done.
Nearest neighbor pattern classification
Thomas M. Cover,Peter E. Hart +1 more
TL;DR: The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points, so it may be said that half the classification information in an infinite sample set is contained in the nearest neighbor.
•Book
Questionnaire Design, Interviewing and Attitude Measurement
Abraham Naftali Oppenheim
- 01 Jul 1992
TL;DR: The second edition of Dr Bram Oppenheim's established work, like the first, is a practical teaching text of survey methods as mentioned in this paper, which includes interviewing (both clip-board and depth interviewing), sampling and research design, data analysis, and a special chapter on pilot work.
5.2K
CUTE: a concolic unit testing engine for C
Koushik Sen,Darko Marinov,Gul Agha +2 more
- 01 Sep 2005
TL;DR: In this paper, the authors address the problem of automating unit testing with memory graphs as inputs, and develop a method to represent and track constraints that capture the behavior of a symbolic execution of a unit with memory graph as inputs.