Test suite

Abstract: Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research. Copyright © 2010 John Wiley & Sons, Ltd.

Abstract: : This research organizes, presents, and analyzes contemporary Multiobjective Evolutionary Algorithm (MOEA) research and associated Multiobjective Optimization Problems (MOPs). Using a consistent MOEA terminology and notation, each cited MOEAs' key factors are presented in tabular form for ease of MOEA identification and selection. A detailed quantitative and qualitative MOEA analysis is presented, providing a basis for conclusions about various MOEA-related issues. The traditional notion of building blocks is extended to the MOP domain in an effort to develop more effective and efficient MOEAs. Additionally, the MOEA community's limited test suites contain various functions whose origins and rationale for use are often unknown. Thus, using general test suite guidelines appropriate MOEA test function suites are substantiated and generated. An experimental methodology incorporating a solution database and appropriate metrics is offered as a proposed evaluation framework allowing absolute comparisons of specific MOEA approaches. Taken together, this document's classifications, analyses, and new innovations present a complete, contemporary view of current MOEA "state of the art" and possible future research. Researchers with basic EA knowledge may also use part of it as a largely self-contained introduction to MOEAs.

Abstract: Empirical studies in software testing research may not be comparable, reproducible, or characteristic of practice. One reason is that real bugs are too infrequently used in software testing research. Extracting and reproducing real bugs is challenging and as a result hand-seeded faults or mutants are commonly used as a substitute. This paper presents Defects4J, a database and extensible framework providing real bugs to enable reproducible studies in software testing research. The initial version of Defects4J contains 357 real bugs from 5 real-world open source pro- grams. Each real bug is accompanied by a comprehensive test suite that can expose (demonstrate) that bug. Defects4J is extensible and builds on top of each program’s version con- trol system. Once a program is configured in Defects4J, new bugs can be added to the database with little or no effort. Defects4J features a framework to easily access faulty and fixed program versions and corresponding test suites. This framework also provides a high-level interface to common tasks in software testing research, making it easy to con- duct and reproduce empirical studies. Defects4J is publicly available at http://defects4j.org.

Abstract: List of Figures. List of Tables. List of Procedures. Foreword. Preface. Acknowledgments. I. PRELIMINARIES. 1. A Small Challenge. 2. How to Use This Book. Reader Guidance. Conventions. FAQs for Object-oriented Testing. Test Process. 3. Testing: A Brief Introduction. What Is Software Testing? Definitions. The Limits of Testing. What Can Testing Accomplish? Bibliographic Notes. 4. With the Necessary Changes: Testing and Object-oriented Software. The Dismal Science of Software Testing. Side Effects of the Paradigm. Language-specific Hazards. Coverage Models for Object-oriented Testing. An OO Testing Manifesto. Bibliographic Notes. II. MODELS. 5. Test Models. Test Design and Test Models. Bibliographic Notes. 6. Combinational Models. How Combinational Models Support Testing. How to Develop a Decision Table. Deriving the Logic Function. Decision Table Validation. Test Generation. Choosing a Combinational Test Strategy. Bibliographic Notes. 7. State Machines. Motivation. The Basic Model. The FREE State Model. State-based Test Design. Bibliographic Notes. 8. A Tester's Guide to the UML. Introduction. General-purpose Elements. Use Case Diagram. Class Diagram. Sequence Diagram. Activity Diagram. Statechart Diagram. Collaboration Diagram. Component Diagram. Deployment Diagram. Graphs, Relations, and Testing. Bibliographic Notes. III. PATTERNS. 9. Results-oriented Test Strategy. Results-oriented Testing. Test Design Patterns. Test Design Template. Documenting Test Cases, Suites, and Plans. Bibliographic Notes. 10. Classes. Class Test and Integration. Preliminaries. Method Scope Test Design Patterns. Category-Partition. Combinational Function Test. Recursive Function Test. Polymorphic Message Test. Class Scope Test Design Patterns. Invariant Boundaries. Nonmodal Class Test. Quasi-modal Class Test. Modal Class Test. Flattened Class Scope Test Design Patterns. Polymorphic Server Test. Modal Hierarchy Test. Bibliographic Notes. 11. Reusable Components. Testing and Reuse. Test Design Patterns. Abstract Class Test. Generic Class Test. New Framework Test. Popular Framework Test. Bibliographic Notes. 12. Subsystems. Subsystems. Subsystem Test Design Patterns. Class Association Test. Round-trip Scenario Test. Controlled Exception Test. Mode Machine Test. Bibliographic Notes. 13. Integration. Integration in Object-oriented Development. Integration Patterns. Subsystem/System Scope. Big Bang Integration. Bottom-up Integration. Top-down Integration. Collaboration Integration. Backbone Integration. Layer Integration. Client/Server Integration. Distributed Services Integration. High-frequency Integration. Bibliographic Notes. 14. Application Systems. Testing Application Systems. Test Design Patterns. Extended Use Case Test. Covered in CRUD. Allocate Tests by Profile. Implementation-specific Capabilities. Post-development Testing. Note on Testing Performance Objectives. Bibliographic Notes. 15. Regression Testing. Preliminaries. Test Patterns. Retest All. Retest Risky Use Cases. Retest by Profile. Retest Changed Code. Retest Within Firewall. Bibliographic Notes. IV. TOOLS. 16. Test Automation. Why Testing Must Be Automated. Limitations and Caveats. 17. Assertions. Introduction. Implementation-based Assertions. Responsibility-based Assertions. Implementation. The Percolation Pattern. Deployment. Limitations and Caveats. Some Assertion Tools. Bibliographic Notes. 18. Oracles. Introduction. Oracle Patterns. Comparators. Bibliographic Notes. 19. Test Harness Design. How to Develop a Test Harness. Test Case Patterns. Test Case/Test Suite Method. Test Case/Test Suite Class. Catch All Exceptions. Test Control Patterns. Server Stub. Server Proxy. Driver Patterns. TestDriver Superclass. Percolate the Object Under Test. Symmetric Driver. Subclass Driver. Private Access Driver. Test Control Interface. Drone. Built-in Test Driver. Test Execution Patterns. Command Line Test Bundle. Incremental Testing Framework. Fresh Objects. A Test Implementation Syntax. Bibliographic Notes. Appendix. BigFoot's Tootsie: A Case Study. Requirements. OOA/D for Capability-driven Testing. Implementation. Glossary. References. Index. 0201809389T04062001