Abstract: Unit testing represents a key activity in software development and maintenance. Test suites with high internal quality facilitate maintenance activities, such as code comprehension and regression testing. Several guidelines have been proposed to help developers write good test suites. Unfortunately, such rules are not always followed resulting in the presence of bad test code smells (or simply test smells). Test smells have been defined as poorly designed tests and their presence may negatively affect the maintainability of test suites and production code. Despite the many studies that address code smells in general, until now there has been no empirical evidence regarding test smells (i) distribution in software systems nor (ii) their impact on the maintainability of software systems. This paper fills this gap by presenting two empirical studies. The first study is an exploratory analysis of 18 software systems (two industrial and 16 open source) aimed at analyzing the distribution of test smells in source code. The second study, a controlled experiment involving twenty master students, is aimed at analyzing whether the presence of test smells affects the comprehension of source code during software maintenance. The results show that (i) test smells are widely spread throughout the software systems studied and (ii) most of the test smells have a strong negative impact on the comprehensibility of test suites and production code.

Abstract: We investigate how to execute a unit test for all products of a product line without generating each product in isolation in a brute-force fashion. Learning from variability-aware analyses, we (a) design and implement a variability-aware interpreter and, alternatively, (b) reencode variability of the product line to simulate the test cases with a model checker. The interpreter internally reasons about variability, executing paths not affected by variability only once for the whole product line. The model checker achieves similar results by reusing powerful off-the-shelf analyses. We experimented with a prototype implementation for each strategy. We compare both strategies and discuss trade-offs and future directions. In the long run, we aim at finding an efficient testing approach that can be applied to entire product lines with millions of products.

Abstract: In this paper, we investigate empirically the relationship between object-oriented design metrics and testability of classes. We address testability from the point of view of unit testing effort. We collected data from three open source Java software systems for which JUnit test cases exist. To capture the testing effort of classes, we used metrics to quantify the corresponding JUnit test cases. Classes were classified, according to the required unit testing effort, in two categories: high and low. In order to evaluate the relationship between object-oriented design metrics and unit testing effort of classes, we used logistic regression methods. We used the univariate logistic regression analysis to evaluate the individual effect of each metric on the unit testing effort of classes. The multivariate logistic regression analysis was used to explore the combined effect of the metrics. The performance of the prediction models was evaluated using Receiver Operating Characteristic analysis. The results indicate that: 1) complexity, size, cohesion and (to some extent) coupling were found significant predictors of the unit testing effort of classes and 2) multivariate regression models based on object-oriented design metrics are able to accurately predict the unit testing effort of classes.

Abstract: Learn to build fast and scalable software in JavaScript with Node.jsNode.js is a powerful and popular new framework for writing scalable network programs using JavaScript. This no nonsense book begins with an overview of Node.js and then quickly dives into the code, core concepts, and APIs. In-depth coverage pares down the essentials to cover debugging, unit testing, and flow control so that you can start building and testing your own modules right away.Covers node and asynchronous programming main conceptsAddresses the basics: modules, buffers, events, and timersExplores streams, file systems, networking, and automated unit testingGoes beyond the basics, and shares techniques and tools for debugging, unit testing, and flow control If you already know JavaScript and are curious about the power of Node.js, then this is the ideal book for you.

Abstract: Unit and integration tests can be invaluable during software maintenance as they help to understand pieces of code, they help with quality assurance and they build up confidence amongst developers. Unfortunately then, previous research has shown that unit tests do not always co-evolve nicely with the production code, thus leaving the software vulnerable. This paper presents Test N Force, a tool that helps developers to identify the unit tests that need to be altered and executed after a code change, thereby reducing the effort needed to keep the unit tests in sync with the changes to the production code. In order to evaluate Test N Force, we perform a user study that evaluates the adequacy, usefulness and completeness of Test N Force.

Abstract: A computer-implemented method and apparatus for unit testing, the method comprising: intercepting user interactions when a user is testing a function implemented by user code; recording execution data for the function execution; generating one or more tests based on the execution data, wherein each test is generated in accordance with a heuristic applicable for the function; detecting a code change in the function; and automatically executing the tests function to test the user code as changed.

Abstract: Operating systems of modern mobile devices, like e.g. iOS and Android, require the applications to conform to a life cycle model, to ensure the functional correctness of the application and its data integrity over exceptional behavior as e.g. out-swapping of the application. The applications life cycle events are triggered asynchronously by the system and depend on the environment. In order to test life cycle dependent properties of the applications, we define a unit testing based approach that uses life cycle callback-methods. The method identifies life cycle dependent properties in the application specification, and derives assertion-based test cases for validating the conformance of the properties. Life cycle triggers are used in the test case execution. The paper describes to which application features the approach can be applied, and the limitations of the approach. A case study demonstrates how to apply our approach to state-of-the-art mobile platforms, using Android 2.2 as an example.

Abstract: Compared to functional unit testing, automated performance testing is difficult, partially because correctness criteria are more difficult to express for performance than for functionality. Where existing approaches rely on absolute bounds on the execution time, we aim to express assertions on code performance in relative, hardware-independent terms. To this end, we introduce Stochastic Performance Logic (SPL), which allows making statements about relative method performance. Since SPL interpretation is based on statistical tests applied to performance measurements, it allows (for a special class of formulas) calculating the minimum probability at which a particular SPL formula holds. We prove basic properties of the logic and present an algorithm for SAT-solver-guided evaluation of SPL formulas, which allows optimizing the number of performance measurements that need to be made. Finally, we propose integration of SPL formulas with Java code using higher-level performance annotations, for performance testing and documentation purposes.

Abstract: Platforms such as Windows Azure let applications conduct data-intensive cloud computing. Unit testing can help ensure high-quality development of such applications, but the results depend on test inputs and the cloud environment's state. Manually providing various test inputs and cloud states is laborious and time-consuming. However, automated test generation must simulate various cloud states to achieve effective testing. To address this challenge, a proposed approach models the cloud environment and applies dynamic symbolic execution to generate test inputs and cloud states. Applying this approach to open-source Azure cloud applications shows that it can achieve high structural coverage.

Abstract: T his article studies how CS1 students responded to two recent trends in programming education: TDD-like testing, which we applied to procedural programming tasks, and game contextualization. Our main conclusions are: (1) to make students realize the importance of test writing, we need to design more programming tasks where TDD-like tests are relevant. (2) Merely working with a simple game programming library can inhibit the learning of basic programming concepts. (3) While the game development (course component) motivated students, they also reported that teacher support and well-functioning course arrangements are very important. learning properly. Rather, the students need to be able to review ready-made tests, preferably writing their own tests, to benefit from TDD in problem solving. They also point out that designing tests first was a particularly suitable strategy for weaker students, as tests tend to be simpler to start when compared to the desired functionalities. Overall, the claims advanced for the educational benefits of TDD and unit testing in the CS1 context appear to amount to students' improved understanding of program behavior. A current CS1 trend, perhaps a more topical one than TDD and unit testing, is game contextualization. Studies where games have had one role or another in CS1 have consistently reported positive experiences. Games can attract both males and females [12, 15], and students become confident in their own learning abilities with games [12]. Games appear to have a good fit with object-oriented programming, leading to a natural analysis under the OO approach [11]. Graphical games tend to match well with the constructivist view of learning, as the visual experimentation (that is involved) is likely to help in developing internal models of programming concepts [15]. Games provide meaningful study content as students can share their games within their own social network [19]. Some experiences indicate that the use of games has clearly improved retention numbers without sacrificing any technical depth of programming [12]. Improved retention does not always emerge [1], while improved student performance is quite often reported. The paper by Rajaja-vivarma [19] suggests some reasons for this improved performance. That is, with games students develop tenacity in their learning efforts , and consequently can overcome many known beginner challenges. For example, students unaffectedly focus on design questions and keep testing their products until they work properly. The complexity of implementing attractive games is regarded as the key challenge of game development in introductory programming courses …

Abstract: In order to support test suite understanding, we investigate whether we can automatically derive relations between test cases. In particular, we search for trace-based similarities between (high-level) end-to-end tests on the one hand and fine grained unit tests on the other. Our approach uses the shared word count metric to determine similarity. We evaluate our approach in two case studies and show which relations between end-to-end and unit tests are found by our approach, and how this information can be used to support test suite understanding.

Abstract: The invention relates to a construction method of an airborne computer software testing general system, which comprises five steps: 1, writing a testing plan file and establishing a testing system; 2, carrying out a unit test (static) during code development; 3, carrying out a unit/integration test (dynamic) by an actual operation function module; 4, carrying out a system test after the unit/integration test; and 5, carrying out an acceptance test. The invention provides the airborne computer software testing general system which achieves the testing purpose economically and effectively via establishing a testing system collateral with the code development, and enhances the normalization and reliability of the software system, and the software testing personal can completely and efficiently deploy and execute testing work according to specific needs. The invention has practical value and broad application prospects in the technical field of the airborne computer software test.

Abstract: We present BloomUnit, a testing framework for distributed programs written in the Bloom language. BloomUnit allows developers to write declarative test specifications that describe the input/output behavior of a software module. Test specifications are expressed as Bloom queries over (distributed) execution traces of the program under test. To allow execution traces to be produced automatically, BloomUnit synthesizes program inputs that satisfy user-provided constraints. For a given input, BloomUnit systematically explores the space of possible network message reorderings. BloomUnit searches this space efficiently by exploiting program semantics to ignore "uninteresting" message schedules.We illustrate the utility of BloomUnit by demonstrating an incremental process by which a programmer might provide and refine a set of queries and constraints until they define a rich set of correctness tests for a distributed system.

Abstract: According to one aspect of the invention, a system and method for unit and regression testing has been developed. The system is capable to automate unit/regression test case preparation, remote execution, provide XML based assert mechanism to evaluate test results, maintenance and versioning. The present system enables model driven testing of domain specific languages and provides a user friendly mechanism to perform Unit and Regression Testing.

Abstract: The aim of this paper is to evaluate empirically the relationship between a new metric (Quality Assurance Indicator--Qi) and testability of classes in object-oriented systems. The Qi metric captures the distribution of the control flow in a system. We addressed testability from the perspective of unit testing effort. We collected data from five open source Java software systems for which JUnit test cases exist. To capture the testing effort of classes, we used different metrics to quantify the corresponding JUnit test cases. Classes were classified, according to the required testing effort, in two categories: high and low. In order to evaluate the capability of the Qi metric to predict testability of classes, we used the univariate logistic regression method. The performance of the predicted model was evaluated using Receiver Operating Characteristic (ROC) analysis. The results indicate that the univariate model based on the Qi metric is able to accurately predict the unit testing effort of classes.

Abstract: There continues to be a lack of adequate training for students in software testing techniques and tools at most academic institutions. Several educators and researchers have investigated innovative approaches that integrate testing into programming and software engineering (SE) courses with some success. The main problems are getting other educators to adopt their approaches and ensuring students continue to use the techniques they learned in previous courses. In this paper we present a study that evaluates a non-intrusive approach to integrating software testing techniques and tools in SE courses. The study uses a Web-Based Repository of Software Testing Tools (WReSTT) that contains tutorials on software testing concepts and tools. The results of the study show that (1) students who use WReSTT in the classroom can improve their understanding and use of testing techniques and tools, (2) students find WReSTT a useful learning resource, and (3) the collaborative learning environment motivates students to complete assignments.

Abstract: Several approaches exist to automatically derive test cases that check the conformance of the implementation of abstract data types (ADTs) with respect to their specification. However, they lack support for the testing of implementations of ADTs defined by generic classes. In this paper, we present a novel technique to automatically derive, from specifications, unit test cases for Java generic classes that, in addition to the usual testing data, encompass implementations for the type parameters. The proposed technique relies on the use of Alloy Analyzer to find model instances for each test goal. JUnit test cases and Java implementations of the parameters are extracted from these model instances.

Abstract: During execution of a unit test, receiving from the unit test a first request referencing a mock object. An instance of the mock object and initial cached mock object data is returned to the test unit, wherein the initial cached mock object data includes first data for a real object represented by the mock object. Second data for the real object represented by the mock object is collected. The second data for the real object is compared to the initial cached mock object data. Responsive to determining that at least one aspect of the second data for the real object does not correspond to the initial cached mock object data, the cached mock object data is updated with the second data for the real object. An indication can be provided to the unit test that the initial cached mock object data returned to the unit test is unreliable.

Abstract: While past research in answer-set programming (ASP) mainly focused on theory, ASP solver technology, and applications, the present work situates itself in the context of a quite recent research trend: development support for ASP. In particular, we propose to augment answer-set programs with additional meta-information formulated in a dedicated annotation language, called Lana. This language allows the grouping of rules into coherent blocks and to specify language signatures, types, pre-and postconditions, as well as unit tests for such blocks. While these annotations are invisible to an ASP solver, as they take the form of program comments, they can be interpreted by tools for documentation, testing, and verification purposes, as well as to eliminate sources of common programming errors by realising syntax checking or code completion features. To demonstrate its versatility, we introduce two such tools, viz. (i) ASPDoc, for generating an HTML documentation for a program based on the annotated information, and (ii) ASPUnit, for running and monitoring unit tests on program blocks. Lana is also exploited in the SeaLion system, an integrated development environment for ASP based on Eclipse.

Abstract: While past research in answer-set programming (ASP) mainly focused on theory, ASP solver technology, and applications, the present work situates itself in the context of a quite recent research trend: development support for ASP. In particular, we propose to augment answer-set programs with additional meta-information formulated in a dedicated annotation language, called LANA. This language allows the grouping of rules into coherent blocks and to specify language signatures, types, pre- and postconditions, as well as unit tests for such blocks. While these annotations are invisible to an ASP solver, as they take the form of program comments, they can be interpreted by tools for documentation, testing, and verification purposes, as well as to eliminate sources of common programming errors by realising syntax checking or code completion features. To demonstrate its versatility, we introduce two such tools, viz. (i) ASPDOC, for generating an HTML documentation for a program based on the annotated information, and (ii) ASPUNIT, for running and monitoring unit tests on program blocks. LANA is also exploited in the SeaLion system, an integrated development environment for ASP based on Eclipse. To appear in Theory and Practice of Logic Programming

Abstract: Test driven development (TDD) is the practice of writing unit tests before writing the source. TDD practitioners typically start with example-based unit tests to verify an understanding of the software's intended functionality and to drive software design decisions. Hence, the typical role of test cases in TDD leans more towards specifying and documenting expected behavior, and less towards detecting faults. Conversely, traditional criteria-based test coverage ignores functionality in favor of tests that thoroughly exercise the software. This paper examines whether it is possible to combine both approaches. Specifically, can additional criteria based tests improve the quality of TDD test suites without disrupting the TDD development process? This paper presents the results of an observational study that generated additional criteria-based tests as part of a TDD exercise. The criterion was mutation analysis and the additional tests were designed to kill mutants not killed by the TDD tests. The additional unit tests found several software faults and other deficiencies in the software. Subsequent interviews with the programmers indicated that they welcomed the additional tests, and that the additional tests did not inhibit their productivity.

Abstract: The utility model discloses a merging unit test system with analog input and university, which comprises a tester for receiving a merging unit to be tested and a signal outputted by an analog signal source as well as a host computer for running a background analysis system, wherein the host computer is connected and communicated with the tester through an interface. The merging unit test system adopts an independent high-precision collection module, so that the high universality of the test system is ensured, the test system can be freely connected and in butt joint with the merging units of different manufacturers, and strong universality and wide applicability can be realized; and due to the adoption of the layered portable structure design, the problems of the electromagnetic compatibility, heat radiation, transportation, carrying and the like of the test instrument can be well solved, and the urgent need of the development of a smart grid can be met.

Abstract: A method, computer program product, and system for transforming unit tests is described. A unit test associated with one or more software units is identified. A first input parameter of the unit test is identified. A substitute parameter value is determined, wherein the substitute parameter value is associated with a security test for the one or more software units. A value of the first input parameter in the unit test is replaced with the substitute parameter value. The unit test including the substitute parameter value is implemented for the one or more software units. A first security issue associated with the one or more software units is identified, based upon, at least in part, replacing the first input parameter of the unit test with the substitute parameter value and implementing the unit test including the substitute parameter value.

Abstract: IEC 61850 is an international standard for communication networks for substation [1]. As a part of the IEC 61850 series, IEC 61850 Part 9-2 was introduced in 2004 and updated to a stable version in 2011 [2]. Part 9-2 de nes the specfic communication service mapping for the transmission of sampled values from process equipment [3]. The interface between process equipment and bay level devices is named as Merging Unit (MU) [4]. The 9-2 standard is a newly updated standard which has not yet been implemented in large scale. Therefore it is new or unfamiliar to many engineers working with substation automation systems. A project which is aimed to develop a light weight IEC 61850-9-2 MU testing environment is conducted in ICS department in KTH. The project can be also referred as "soft" MU project and consists of two parts. Part A is to implement conventional power system model, instrument transformer, and analog to digital converter [5]. This thesis project is Part B of the light weight merging unit (MU) testing tool project. This project is focusing on developing the 9-2 process bus communication interface for the "soft" MU project. The digitalized measurements such as current and voltage from Project A are encoded into 9-2 sampled values (SV) Ethernet stream by the process bus interface. The project is executed in the following steps. Firstly, the interface is programmed by C language under Linux environment. Secondly, the process bus communication interface is embedded into Matlab so that Part A and Part B can be combined as one system. To evaluate the light weight IEC 61850-9-2 MU testing environment, the "soft" MU is connected to a process bus network with one IED. The IED used in this thesis project is RET 670 which is a product of ABB Substation Automation. The four steps over current protection and two Windings transformer dierential protection in RET 670 is tested. The evaluation work is also demonstrated in this report.

Abstract: The JUnit Generation (JUG) system provides fast, semiautomated feedback to students. It uses a Java-like script to generate unit tests and time complexity tests, then runs those tests to generate reports. The goals for JUG are improved feedback for students, and decreased preparation and grading time for instructors and grading assistants.

Abstract: This book explains in detail how to implement unit tests using two very popular open source Java technologies: TestNG and Mockito. It presents a range of techniques necessary to write high quality unit tests - e.g. mocks, parametrized tests and matchers. It also discusses trade-offs related to the choices we have to make when dealing with some real-life code issues. The book stresses the importance of writing readable and maintainable unit tests, and puts a lot of stress on code quality. It shows how to achieve testable code and to eliminate common mistakes by following the Test Driven Development approach. Every topic discussed in the book is illustrated with code examples, and each chapter is accompanied by some exercises. By reading this book you will: Grasp the role and purpose of unit tests Write high-quality, readable and maintainable unit tests Learn how to use TestNG and Mockito (but also other useful tools) Avoid common pitfalls when writing unit tests Recognize bad unit tests, and fix them in no time Develop code following the Test Driven Development (TDD) approach Use mocks, stubs and test-spies intelligently Measure the quality of your tests using code coverage and mutation testing Learn how to improve your tests' code so it is an asset and not a burden Test collections, expected exceptions, time-dependent methods and much more Customize test reports so that they show you what you really need to know Master tools and techniques your team members have never even heard of (priceless!) :) Nowadays every developer is expected to write unit tests. While simple in theory, in practice writing high-quality unit tests can turn out to be a real challenge. This book will help.

Abstract: There is provided a function test apparatus based on unit test cases reusing and a function test method thereof. The function test apparatus related to the present disclosure is constructed with a Storage Portion, storing a unit test case for each function into a hierarchical constitution; a Selection Portion of unit test case, when a Test Object Function is provided, in order to abstract an internal constitution of a callee function activated by the Test Object Function, selectively reusing a unit test case of the callee function; a Generator of test case, generating at least one test case in order for all the unit test cases selected by the Selection Portion of test case to be performed; and a Performance Portion of test case, testing the Test Object Function by performing the test case. According to the configuration herein, a test for function may be conducted without analyzing the internal constitution of software function at an integral testing motion and a test of which a Test coverage is high may be conducted to the Test Object Function. Therefore, an advantage of high reliability for the test result is obtained.