Ranking programs using black box testing
Koen Claessen,John Hughes,Michal H. Palka,Nicholas Smallbone,Hans Svensson +4 more
- 03 May 2010
- pp 103-110
TL;DR: An unbiased method for measuring the relative quality of different solutions to a programming problem based on identifying possible bugs from program behaviour through black-box testing, which was aimed at evaluating the effectiveness of property-based testing vs. unit testing in software development.
read more
Abstract: We present an unbiased method for measuring the relative quality of different solutions to a programming problem. Our method is based on identifying possible bugs from program behaviour through black-box testing. The main motivation for such a method is its use in experimental evaluation of software development methods. We report on the use of our method in a small-scale such experiment, which was aimed at evaluating the effectiveness of property-based testing vs. unit testing in software development.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Software testing with QuickCheck
TL;DR: This paper presents a tutorial, with extensive exercises, in the use of Quviq QuickCheck--a property-based testing tool for Erlang, which enables developers to formulate formal specifications of their code and to use them for testing.
31
PrologCheck – Property-Based Testing in Prolog
Cláudio Amaral,Mário Florido,Vítor Santos Costa +2 more
- 04 Jun 2014
TL;DR: The tool is inspired by the well known QuickCheck, originally designed for the functional programming language Haskell and includes features that deal with specific characteristics of Prolog such as its relational nature and the absence of a strong type discipline.
Automatic Grading of Programming Exercises using Property-Based Testing
Clara Benac Earle,Lars-Åke Fredlund,John Hughes +2 more
- 11 Jul 2016
TL;DR: A framework for automatic grading of programming exercises using property-based testing, a form of model-based black-box testing, that considers code structural metrics, such as McCabe's cyclomatic complexity, giving rise to a composite program grade that includes both functional, non-functional, and code structural aspects.
22
Teaching Property-Based Testing: Why and How
Isabel Azevedo,Nuno Malheiro +1 more
- 01 Jan 2020
TL;DR: Property-based testing has been taught in the Advanced Programming Techniques course of the master's program in Informatics Engineering of the Instituto Superior de Engenharia do Porto to give students tools that can increase the quality of their production as software developers.
2
References
•Book
Experimentation in Software Engineering: An Introduction
Claes Wohlin,Per Runeson,Martin Höst,Magnus C. Ohlsson,Bjöorn Regnell,Anders Wesslén +5 more
- 27 Sep 2011
TL;DR: The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering, and to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering.
3.1K
Preliminary guidelines for empirical research in software engineering
Barbara Kitchenham,Shari Lawrence Pfleeger,Lesley M. Pickard,Peter W. Jones,D.C. Hoaglin,K. El Emam,J. Rosenberg +6 more
TL;DR: A preliminary set of research guidelines aimed at stimulating discussion among software researchers, intended to assist researchers, reviewers, and meta-analysts in designing, conducting, and evaluating empirical studies.
QuickCheck: a lightweight tool for random testing of Haskell programs
Koen Claessen,John Hughes +1 more
- 01 Sep 2000
TL;DR: QuickCheck is a tool which aids the Haskell programmer in formulating and testing properties of programs, and can be automatically tested on random input, but it is also possible to define custom test data generators.
Pex: white box test generation for .NET
Nikolai Tillmann,Jonathan de Halleux +1 more
- 09 Apr 2008
TL;DR: Pex automatically produces a small test suite with high code coverage for a .NET program by performing a systematic program analysis using dynamic symbolic execution, similar to path-bounded model-checking, to determine test inputs for Parameterized Unit Tests.
Software Testing Research: Achievements, Challenges, Dreams
Antonia Bertolino
- 23 May 2007
TL;DR: A consistent roadmap of the most relevant challenges to be addressed in software testing research is proposed, constituted by some important past achievements, while the destination consists of four identified goals to which research ultimately tends, but which remain as unreachable as dreams.
Related Papers (5)
Bo Zhang,Xiangheng Shen,Jun-jie Wang,Yuan Chen +3 more
- 13 Mar 2013
Pu Yunming,Fan Ming-hong +1 more
- 25 Nov 2008