Abstract: Unit testing is testing of individual units or groups of related units. What are a company's typical strengths and weaknesses when applying unit testing? Per Beremark and the author surveyed unit testing practices on the basis of focus group discussions in software process improvement network (SPIN) and launched a questionnaire to validate the results. This survey is an indication of unit testing in several companies. You can use the questionnaire at your own company to clarify what you mean by unit testing, to identify the strengths and weaknesses of your unit testing practices, and to compare with other organizations to improve those practices

Abstract: Unit tests are becoming popular. Are there ways to automate the generation of good unit tests? Parameterized unit tests are unit tests that depend on inputs. PUTs describe behavior more concisely than traditional unit tests. We use symbolic execution techniques and constraint solving to find inputs for PUTs that achieve high code coverage, to turn existing unit tests into PUTs, and to generate entirely new PUTs that describe an existing implementation's behavior. Traditional testing benefits from these techniques because test inputs - including the behavior of entire classes - can often be generated automatically from compact PUTs

Abstract: A community for maintaining user membership can be provided. Each major component of the communities framework can have multiple portals associated with its series of tests. Each of these portals can start out as a portal file. These portal files can be ran through the dissassembler in order to render the portals representing individual community instances. Once the community instance is created for the portal, various HTML unit tests can be written by the developer in order to access that portal as some user with given membership and functional capabilities. The test framework can automatically create users giving them membership in the community and mapping them to membership capabilities. The component code can subsequently be accessed in the context of a real user or member of the community. In this manner, a more realistic and efficient testing environment is achieved.

Abstract: Test driven development (TDD) is gaining interest among practitioners and researchers: it promises to increase the quality of the code. Even if TDD is considered a development practice, it relies on the use of unit testing. For this reason, it could be an alternative to the testing after coding (TAC), which is the usual approach to run and execute unit tests after having written the code. We wondered which are the differences between the two practices, from the standpoint of quality and productivity. In order to answer our research question, we carried out an experiment in a Spanish Software House. The results suggest that TDD improves the unit testing but slows down the overall process.

Abstract: There are several problem areas that must be addressed when applying randomization to unit testing. As yet no general, fully automated solution that works for all units has been proposed. We therefore have developed RUTE-J, a Java package intended to help programmers do randomized unit testing in Java. In this paper, we describe RUTE-J and illustrate how it supports the development of per-unit solutions for the problems of randomized unit testing. We report on an experiment in which we applied RUTE-J to the standard Java TreeMap class, measuring the efficiency and effectiveness of the technique. We also illustrate the use of randomized testing in experimentation, by adapting RUTE-J so that it generates randomized minimal covering test suites, and measuring the effectiveness of the test suites generated.

Abstract: Unit testing is a popular way to guide software development and testing. Each unit test should target a single feature, but in practice it is difficult to test features in isolation. Mock objects are a well-known technique to substitute parts of a program which are irrelevant for a particular unit test. Today mock objects are usually written manually supported by tools that generate method stubs or distill behavior from existing programs. We have developed a prototype tool based on symbolic execution of .NET code that generates mock objects including their behavior by analyzing all uses of the mock object in a given unit test. It is not required that an actual implementation of the mocked behavior exists. We are working towards an integration of our tool into Visual Studio Team System.

Abstract: Test-driven development (TDD) is a key practice for agile developers because it involves writing test cases ahead of the code, which can improve design The TDD process works well for projects in which a collocated team develops a small to medium system, but it can be challenging for large systems, especially those involving geographically distributed teams The main obstacle is the degree of integration: when the team must integrate many individual classes developed at distributed sites, the coordination and communication grows exponentially with the number of individual developers and sites This does not mean that TDD is ineffective for large-scale geographically distributed projects, but developers must take care to account for its focus on unit testing and its failure to rigorously address communications issues during system and integration testing In this article, suggestions to scale up TDD are presented with two large-scale global software development projects at a major corporation and a recent meeting to exchange global software development best practices with a Fortune 500 company

Abstract: For many software development organizations it is of crucial importance to reduce development costs while still maintaining high product quality. Since testing commonly constitutes a significant part of the development time, one way to increase efficiency is to find more faults early when they are cheaper to pinpoint and remove. This paper presents empirical results from introducing a concept for early fault detection. That is, an alternative approach to Test-Driven Development which was applied on a component level instead of on a class/method level. The selected method for evaluating the result of introducing the concept was based on an existing method for fault-based process assessment and was proven practically useful for evaluating fault reducing improvements. The evaluation was made on two industrial projects and on different features within a project that only implemented the concept partly. The evaluation result demonstrated improvements regarding decreased fault rates and Return On Investment (ROI), e.g. the total project cost became about 5-6% less already in the first two studied projects.

Abstract: Evolutionary algorithms have been successfully applied in the area of software testing. However, previous approaches in the area of object-oriented testing are limited in terms of test case feasibility due to call dependences and runtime exceptions. In this paper, we present a search-based approach to automatically generating test cases for object-oriented software. It relies on a tree-based representation of method call sequences. Strongly-typed genetic programming is employed to generate method call trees which respect the call dependences among the methods. We apply a new kind of distance-based fitness function that accounts for runtime exceptions. In a case study, the approach outperformed random testing in terms of achieved coverage and it produced test cases achieving full branch coverage for a test object that makes ample use of explicit runtime exceptions.

Abstract: Aspect-oriented software development is gaining popularity with the adoption of languages such as AspectJ. Testing is an important part in any software development, including aspect-oriented development. To automate generation of unit tests for AspectJ programs, we can apply the existing tools that automate generation of unit tests for Java programs. However, these tools can generate a large number of test inputs, and manually inspecting the behavior of the software on all these inputs is time consuming. We propose Raspect, a framework for detecting redundant unit tests for AspectJ programs. We introduce three levels of units in AspectJ programs: advised methods, advice, and intertype methods. We show how to detect at each level redundant test that does not exercise new behavior. Our approach selects only non-redundant tests from the automatically generated test suites, thus allowing the developer to spend less time in inspecting this reduced set of tests. We have implemented Raspect and applied it on 12 subjects taken from a variety of sources; our experience shows that Raspect can effectively reduce the size of generated test suites for inspecting AspectJ programs

Abstract: A system and method for identifying errors in a computer software include: identifying a potential problem in the computer software; triggering a portion of the computer software by the identified potential problem; determining a control flow graph and a data flow graph for the triggered portion of the computer software originating at the identified potential problem; and analyzing the control flow graph and the data flow graph to verify that the identified potential problem is an actual error. The potential problem may be identified using test tools such as a static analysis tool or a unit test tool.

Abstract: An automatic test-generation tool can produce a large number of test inputs to exercise the class under test. However, without specifications, developers cannot inspect the execution of each automatically generated test input practically. To address the problem, we have developed an automatic test abstraction tool, called Abstra, to extract high level object-state-transition information from unit-test executions, without requiring a priori specifications. Given a class and a set of its generated test inputs, our tool extracts object state machines (OSM): a state in an OSM represents an object state of the class and a transition in an OSM represents method calls of the class. When an object state in an OSM is concrete (being represented by the values of all fields reachable from the object), the size of the OSM could be too large to be useful for inspection. To address this issue, we have developed techniques in the tool to abstract object states based on returns of observer methods, branch coverage of methods, and individual object fields, respectively. The tool provides useful object-state-transition information for programmers to inspect unit-test executions effectively. In particular, the tool helps facilitate correctness inspection, program understanding, fault isolation, and test characterization.

Abstract: Software testing is an important technique for assurance of software quality. Mutation testing has been identified as a powerful fault-based technique for unit testing, and there has been some research on automatic generation of test data for mutation testing. However, existing approaches to this kind of test data generation usually generate test data according to one mutant at one time. Thus, more test data that are needed for achieving a given mutation score. In this paper, we propose a new approach to generating one test data according to multiple mutants that are mutated at the same location at one time. Thus, our approach can generate smaller test suite that can achieve the same mutation testing score. To evaluate our approach, we implemented a prototype tool based on our approach and carried out some preliminary experiments. The experimental results show that our approach is more cost-effective.

Abstract: In this paper a methodology for the application of computer simulation to evacuation certification of aircraft is suggested. This involves the use of computer simulation, historic certification data, component testing, and full-scale certification trials. The methodology sets out a framework for how computer simulation should be undertaken in a certification environment and draws on experience from both the marine and building industries. In addition, a phased introduction of computer models to certification is suggested. This involves as a first step the use of computer simulation in conjunction with full-scale testing. The combination of full-scale trial, computer simulation (and if necessary component testing) provides better insight into aircraft evacuation performance capabilities by generating a performance probability distribution rather than a single datum. Once further confidence in the technique is established the requirement for the full-scale demonstration could be dropped. The second step in the adoption of computer simulation for certification involves the introduction of several scenarios based on, for example, exit availability, instructed by accident analysis. The final step would be the introduction of more realistic accident scenarios. This would require the continued development of aircraft evacuation modeling technology to include additional behavioral features common in real accident scenarios.

Abstract: This paper describes a prototype tool, called SimC, which automatically generates test data for unit testing of C programs. The tool symbolically simulates the execution of the given program. It simulates pointer operations precisely. This makes it capable of generating test data for programs involving pointer and structure operations. Experiments on real-world programs including the GNU coreutils are presented. Some trade-offs of simulation schemes are also discussed

Abstract: : CUTE, a Concolic Unit Testing Engine for C and Java, is a tool to systematically and automatically test sequential C programs (including pointers) and concurrent Java programs CUTE combines concrete and symbolic execution in a way that avoids redundant test cases as well as false warnings The tool also introduces a race-flipping technique to efficiently test and model check concurrent programs with data inputs

Abstract: Test-driven learning (TDL) is an approach to teaching computer programming that involves introducing and exploring new concepts through automated unit tests. TDL offers the potential of teaching te...

Abstract: We propose a solution to improve the confidence on the correctness of applications designed to be executed in heterogeneous environments, like a grid. Our solution is motivated by the observation that the traditional ways to qualify test processes are based on code coverage metrics. We believe that this approach is not adequate when dealing with applications that can (and do) fail when interacting with heterogeneous execution environments. Besides code coverage, tests must also cover possible environments. As a solution we propose the utilization of InGriD to describe and deploy test environments and GridUnit to coordinate and monitor the execution of test sets. By combining these two solutions we provide a cost effective way to introduce environmental coverage to our test suites, which is complementary and orthogonal to traditional code coverage metrics. As a case study, we have shown how our solution could be applied to help testing a grid application called MyPhotoGrid, which uses the grid to parallelize the generation of large photograph albums.

Abstract: In recent years, agile development methods have become increasingly popular, and although few people seem willing to go all the way with Extreme Programming, it seems that there is a general consensus today that test-driven development with unit testing is a Good Thing. However, this requires that tests are easy to write, since programmers are generally lazy, and that running the tests is easy and quick and the test results are presented in a concise manner, to make the feedback loop as short as possible. Failing this, it is likely that testing will not be extensively used during development.The concept of a lightweight unit testing framework that fulfils these goals, tailored to a particular programming language, was popularized by the JUnit framework for Java, written by Kent Beck and Erich Gamma. This was based on an earlier framework for Smalltalk called SUnit, by Kent Beck. The ideas in JUnit are easily transferred to any other object-oriented language, and today variants have been written for many different programming languages.We will here present our adaptation of these ideas to Erlang. The EUnit framework provides the usual features of such frameworks: writing tests is very easy indeed, and so is running them. Like most other similar frameworks, we rely on program introspection and naming conventions to reduce the amount of coding necessary to write tests. However, there are also some unusual aspects. Since Erlang is functional, rather than object oriented, it is not possible to use inheritance to provide basic test functionality, nor to use object instantiation to handle things like setup/teardown of contexts for tests. Instead, we base our system on a "language" for describing sets of tests, using mainly lists, tuples, and lambda expressions. We also make rather heavy use of preprocessor macros to allow more compact and readable notation. Because test descriptions are data, they can be easily combined, abstracted over, or even be generated on the fly. Lambda expressions allow subtests to be instantiated with setup/teardown contexts.Furthermore, the parallel and distributed nature of the Erlang language on one hand provides a challenge, but on the other hand gives us enormous power and flexibility. For instance, it is trivial to express in our test description language that a set of tests should be executed by a separate process, or on a specific machine, or all in parallel, or even as a number of subsets of parallel tests, with each subset running on a separate machine. Apart from providing easy distributed job control, this allows us to write unit tests that test the behaviour of parallel and distributed programs, something which otherwise tends to require so much coding for each test as to be impractical.Although EUnit is still under development, we feel that it has great potential, and that it uses some novel ideas that could be used to implement similar frameworks in other functional languages. EUnit is free software under the GNU Lesser General Public License. It has not yet been publicly released as of this writing.

Abstract: In this paper, we conduct an experimental study over two groups of students comprising of undergraduate students (seniors) who develop software using the conventional way of performing unit testing after development and also by extracting test cases before implementation as in Agile Programming. Both groups developed the same software using an incremental and iterative approach. The results showed that the software had less number of faults when developed using Agile Programming. Also, the quality of software was better and the productivity increased. Keywords: Test driven development, agile programming, case study 1. Introduction Test-Driven Development (TDD) is a technique that involves writing test cases first and then implementing the code necessary in order to pass the tests. The goal is to achieve immediate input and thereby construct a program. This technique is heavily emphasized in Agile or Extreme Programming [1, 2, 3]. This process of designing test-cases prior to the implementation is termed as “Test-First” approach. We consider unit-testing only (by the programmers) and have nothing to do with integration or acceptance testing. But we do take into account the number of faults found while SQA performs formal unit-testing, integrating and performing acceptance testing in order to measure the quality of the software produced. The first step in this approach is to quickly add a test, basically just enough code to fail. Then we run our tests, generally all of them but in order to finish the process quickly, may be only subsets of tests are run to make sure that the new test does fail. We then update the code in order to pass the new tests. Now, we again run our tests. If they fail, we again have to update and retest, else we add the next functionality. There are no particular rules to form the test-cases but more tests are added throughout the implementation. Though, refactoring should be performed in agile programming that is programmers alternate between adding new tests and functionalities to improve its consistency. It is done to improve the readability of code or change of design or removal of unwanted code. There are various advantages in employing TDD. Programmers tend to know immediately whether the new feature has been added in accordance with the specifications. The process in performed in steps comprising of small parts and hence easier to manage. Low number of faults are tend to be found during acceptance testing and maintenance can be viewed as another increment or addition of feature which would make it easier. There is no particular design phase and software is built through the process of refactoring. In short, TDD improves programmer productivity and software quality. There have been a number of studies [4, 5, 6, 7] that have been performed to test the effectiveness of TDD and the results give mixed opinions. We perform an experiment with 2 groups of students, one developing software the conventional way of testing it after implementation and the other group through TDD. In both groups, test cases were developed by programmers and regression testing was performed. Only difference is test cases are written prior to implementation and are tested throughout the production in TDD and test cases are written and tested after implementation in the conventional way. Each group consisted of 9 undergraduate students and time period for the whole study was 3 months. We investigate in this paper through experimental studies the promise of “Test-First” strategy emphasized in agile programming.

Abstract: A method and software product for capturing decision data and generating JUnit/NUnit tests for each of a JAVA or C# code unit basis paths. The present software erects a user-interface for guiding a user through the basis paths in their source code and assisting them in creating the test data necessary to exercise each of those paths. At each decision in a given path, the software tells the user that the decision must evaluate either true or false to follow the current path, and it allows them to specify, select, or construct test data that will satisfy this condition. After the user has specified test data for all test paths, the software creates a new class containing a rewritten version of the user's original source code that allows test data to be injected at each decision while still preserving the overall control flow of the original code. The software then creates a complete JUnit or NUnit test suite to test each basis path from the original class in the new class by injecting the test data specified by the user.

Abstract: A method and system for generating test cases for a computer program including a plurality of test units. The method and system execute the computer program; monitor the execution of the computer program to obtain monitored information and generate one or more test cases utilizing the monitored information.

Abstract: Test driven development uses unit tests for driving the design of the code. Mock object is an object that imitates the behavior of an object with which class under test has an association to assist the unit testing. Although many tools for mock object are used in practice, there has been few research on defining the mock object model. For the comparisons of the testing capability of the tools, it is important to define the models. In this paper we represent the models of the existing mock object tools and indicate the limitations of them. In addition we propose a new model to overcome the limitations of the existing tools. With this model, more general behavior of the class under test related to the mock object can be expressed and tested.

Abstract: Thanks to unit test frameworks such as JUnit, unit testing has become a common practice in object-oriented software development. However, its application in business process programming is far from prevalent. Business process unit testing treats an individual process as the unit under test, and tests its internal logic thoroughly by isolating it from the partner processes. This types of testing cannot be done by current web service testing technologies that are black-box based. This paper proposes an approach to unit testing of Business Process Execution Language for Web services (BPEL4WS, or WS-BPEL as the new name), and introduces a tool prototype named BPEL-Unit, which extends JUnit. The key idea of this approach is to transform process interaction via web service invocations to class collaboration via method calls, and then apply object-oriented test frameworks. BPEL-Unit provides the following advantages: allow developers simulate partner processes easily, simplify test case writing, speed test case execution, and enable automatic regression testing. With BPEL-Unit, BPEL process unit testing can be performed in a standardized, unified and efficient way.

Abstract: The present invention enables a component under test to bind to a single component, that is capable of simulating most depended upon services, including the details of their interface, whether or not they return explicit results or cause side effects, and regardless of the state of their implementation. This invention includes features that allow for dynamic reconfiguration to meet the needs of both manual and automated testing, including the ability to control normal and exceptional results, as well as features to support both unit and integration testing.

Abstract: A development environment in which unit tests are prepared. The unit tests run in a host environment for more complete and accurate testing of program units. The development environment uses a test execution shim to access an adapter running in the host environment. The host adapter may invoke other program objects that run specific tests. Such objects may late bind to the unit under test. An accessor class may facilitate late binding to the specific unit under test while allowing tests authored in the development environment to use context sensitive information needed to access the unit under test.

Abstract: Non-determinism of the thread schedule is a well-known problem in concurrent programming. However, other sources of non-determinism exist which cannot be controlled by an application, such as network availability. Testing a program with its communication resources being unavailable is difficult, as it requires a change on the host system, which has to be coordinated with the test suite. Essentially, each interaction of the application with the environment can result in a failure. Only some of these failures can be tested. Our work identifies such potential failures and develops a strategy for testing all relevant outcomes of such actions. Our tool, Enforcer, combines the structure of unit tests, coverage information, and fault injection. By taking advantage of a unit test infrastructure, performance can be improved by orders of magnitude compared to previous approaches. Our tool has been tested on several real-world programs, where it found faults without requiring extra test code.

Abstract: One of the oldest techniques to explain abstract concepts is to provide concrete examples. By explaining an abstract concept with a concrete example people make sure that the concept is understood and remembered. Examples in software can be used both to test the software and to illustrate its functionality. Object-oriented programs are built around the concepts of classes, methods and variables, where methods are the atoms of the functionality. But the meta-models of object-oriented languages do not allow developers to associate runnable and composable examples with these concepts and particularly not with methods. Unit tests on the other hand, assure the quality of the units under test and document them. Not being integrated into the language, unit tests are not linked explicitly to their units under test which makes it unnecessarily dif ficult to use them for documenting, typing and debugging software. In addition they are not composable making it hard to develop higher level test scenarios in parallel with higher level objects. In this thesis we analyze unit tests to learn about implicit dependencies among tests and from tests to the methods under test. We develop a technique to partially order unit tests in terms of their covered methods, which reveals possible redundancies due to the lack of composability. We show how partial orders can be used to debug and to comprehend software. We then develop a taxonomy based on several case studies revealing that a high fraction of unit tests already implicitly focuses on single methods. We show that the rest of the tests can be decomposed into commands focusing on single methods. We build a meta-model based on our findings of analyzing test interdependencies which establishes how tests can be explicitly linked to their method under test and how they can be composed to form higher-level test scenarios. We explain how the problems of missing links between tests and units under test are solved using our meta-model. Furthermore, we implemented the meta-model and a first user interface on top of it to give first evidence of how our model supports the developer.

Abstract: HOL-TestGen is a test environment for specification-based unit testing build upon the proof assistant Isabelle/HOL. While there is considerable skepticism with regard to interactive theorem provers in testing communities, we argue that they are a natural choice for (automated) symbolic computations underlying systematic tests. This holds in particular for the development on non-trivial formal test plans of complex software, where some parts of the overall activity require inherently guidance by a test engineer. In this paper, we present the underlying methods for both black box and white box testing in interactive unit test scenarios. HOL-TestGen can also be understood as a unifying technical and conceptual framework for presenting and investigating the variety of unit test techniques in a logically consistent way.