Abstract: Randomized testing is an effective method for testing software units. The thoroughness of randomized unit testing varies widely according to the settings of certain parameters, such as the relative frequencies with which methods are called. In this paper, we describe Nighthawk, a system which uses a genetic algorithm (GA) to find parameters for randomized unit testing that optimize test coverage. Designing GAs is somewhat of a black art. We therefore use a feature subset selection (FSS) tool to assess the size and content of the representations within the GA. Using that tool, we can reduce the size of the representation substantially while still achieving most of the coverage found using the full representation. Our reduced GA achieves almost the same results as the full system, but in only 10 percent of the time. These results suggest that FSS could significantly optimize metaheuristic search-based software engineering tools.

Abstract: Container classes such as lists, sets, or maps are elementary data structures common to many programming languages. Since they are a part of standard libraries, they are important to test, which led to research on advanced testing techniques targeting such containers and research on comparing testing techniques using such containers. However, these techniques have not been thoroughly compared to simpler techniques such as random testing. We present the results of a larger case study in which we compare random testing with shape abstraction, a systematic technique that showed the best results in a previous study. Our experiments show that random testing is about as effective as shape abstraction for testing these containers, which raises the question whether containers are well suited as a benchmark for comparing advanced testing techniques.

Abstract: Intuition suggests that random testing should exhibit a considerable difference in the number of faults detected by two different runs of equal duration. As a consequence, random testing would be rather unpredictable. This article first evaluates the variance over time of the number of faults detected by randomly testing object-oriented software that is equipped with contracts. It presents the results of an empirical study based on 1215 h of randomly testing 27 Eiffel classes, each with 30 seeds of the random number generator. The analysis of over 6 million failures triggered during the experiments shows that the relative number of faults detected by random testing over time is predictable, but that different runs of the random test case generator detect different faults. The experiment also suggests that the random testing quickly finds faults: the first failure is likely to be triggered within 30 s. The second part of this article evaluates the nature of the faults found by random testing. To this end, it first explains a fault classification scheme, which is also used to compare the faults found through random testing with those found through manual testing and with those found in field use of the software and recorded in user incident reports. The results of the comparisons show that each technique is good at uncovering different kinds of faults. None of the techniques subsumes any of the others; each brings distinct contributions. This supports a more general conclusion on comparisons between testing strategies: the number of detected faults is too coarse a criterion for such comparisons—the nature of faults must also be considered. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract: Methods and systems are disclosed that allow automated testing of an application program that is configured to receive a touch input. A testing mechanism can be configured to identify the touch input that is designed to produce a specified result. The testing mechanism can generate one or more signals simulating the touch input. The testing mechanism can then check the state of the user interface of the application program being tested and determine whether the actual result conforms to the specified result.

Abstract: Automated random testing has shown to be an effective approach to finding faults but still faces a major unsolved issue: how to generate test inputs diverse enough to find many faults and find them quickly. Stateful testing, the automated testing technique introduced in this article, generates new test cases that improve an existing test suite. The generated test cases are designed to violate the dynamically inferred contracts (invariants) characterizing the existing test suite. As a consequence, they are in a good position to detect new faults, and also to improve the accuracy of the inferred contracts by discovering those that are unsound. Experiments on 13 data structure classes totalling over 28,000 lines of code demonstrate the effectiveness of stateful testing in improving over the results of long sessions of random testing: stateful testing found 68.4% new faults and improved the accuracy of automatically inferred contracts to over 99%, with just a 7% time overhead.

Abstract: This work addresses to problems of on-line testing for the computing circuits, which operate with approximate data. On-line testing development passed under action of model of the exact data, which has defined the purpose and reliability of its method only for a case of the exact data. Features of the approximated calculations make an error inessential in most cases. In this condition the on-line testing methods expose a new property to reject the reliable results detecting the inessential errors. This fact makes a problem of low reliability of on-line testing methods in checking the result. The ways to increase a reliability of the on-line testing methods in checking the result using the features of approximate calculations are shown. New on-line testing methods based on these ways is offered. Residue checking methods for truncated operations with mantissas, the methods using natural information redundancy of arithmetic results and the methods estimating size of errors are developed.

Abstract: Answer-set programming (ASP) is an established paradigm for declarative problem solving, yet comparably little work on testing of answer-set programs has been done so far. In a recent paper, foundations for structure-based testing of answer-set programs building on a number of coverage notions have been proposed. In this paper, we develop a framework for testing answer-set programs based on this work and study how good the structure-based approach to test input generation is compared to random test input generation. The results indicate that random testing is quite ineffective for some benchmarks, while structurebased techniques catch faults with a high ratemore consistently also in these cases.

Abstract: Statistical testing generates test inputs by sampling from a probability distribution that is carefully chosen so that the inputs exercise all parts of the software being tested. Sets of such inputs have been shown to detect more faults than test sets generated using traditional random and structural testing techniques. Search-based statistical testing employs a metaheuristic search algorithm to automate the otherwise labour-intensive process of deriving the probability distribution. This paper proposes an enhancement to this search algorithm: information obtained during fitness evaluation is used to direct the mutation operator to those parts of the representation where changes may be most beneficial. A principled empirical evaluation demonstrates that this enhancement leads to a significant improvement in algorithm performance, and so increases both the cost-effectiveness and scalability of search-based statistical testing. As part of the empirical approach, we demonstrate the use of response surface methodology as an effective and objective method of tuning algorithm parameters, and suggest innovative refinements to this methodology.

Abstract: In this paper, we propose a particle swarm optimization (PSO) based hybrid testing technique named as "pseudo dynamic testing" to generate test data for C programs by fulfilling one of the most demanding test adequacy criteria: the all-path testing criterion using an interpreter The proposed methodology attempts to solve many of the structural testing problems such as dynamic variables, input dependent array index, abstract function calls, infeasible paths and loop handling The key algorithms and heuristics are given which are easy enough to implement, scalable and effective The technique is employed on real world programs to show the robustness of this technique The set of test inputs generated are not redundant as each leads to a different program path

Abstract: Verification of highly configurable systems poses a significant challenge, the challenge of knowing that every configuration works when there often are intractably many different configurations. When a homogeneous abstraction layer has many mutually exclusive alternative implementations, we might, according to the polymorphic server test pattern, test these implementations using one test suite targeted towards the abstraction layer which is then run for each concrete implementation of the abstraction layer. But, the pattern does not handle interaction testing. Combinatorial interaction testing is one of the more promising techniques for doing interaction testing of a software product line. The bow tie testing pattern describes how the configurations which differ only in the implementation layer require one test suite. In addition, comparing the execution results of one product with another provides for a test oracle. The pattern reduces the effort of testing a highly configurable system without reducing the error detection capabilities provided by ordinary combinatorial interaction testing. We present an example of a subset of the Eclipse IDE product line, and show that only 20 test suites is required to test 41 products, a significant reduction.

Abstract: Software testing is an expensive process, which is vital in the industry. Construction of the test-data in software testing requires the major cost and knowing which method to use in order to generate the test data is very important. This paper discusses the performance of search-based algorithms (preferably genetic algorithm) versus random testing, in software test-data generation. A factorial experiment is designed so that, we have more than one factor for each experiment we make. Although many researches have been done in the area of automated software testing, this research differs from all of them due to sample programs (SUTs) which are used. Since the program generation is automatic as well, Grammatical Evolution is used to guide the program generations. They are not goal based, but generated according to the grammar we provide, with different levels of complexity. Genetic algorithm is first applied to programs, then we apply random testing. Based on the results which come up, this paper recommends one method to use for software testing, if the SUT has the same conditions as we had in this study. SUTs are not like the sample programs, provided by other studies since they are generated using a grammar.

Abstract: Testing the interactions of different applications running in the same operating system or platform poses challenges for manual testing and conventional script-based automation. Towards this end, we have developed an online model-based testing solution allowing efficient testing of such interactions. This paper presents the results of the comparison of algorithms used for generating tests for interaction testing. The comparison is based on our experiments with a number of different algorithms as well as results from the earlier studies by others. Given the simplicity of implementation, Random Walk seems very useful and practical solution for online test generation. However, each of the compared algorithms has its strong points, making the selection dependent of the metric one wants to emphasize and the available a priori information. Especially when the execution of test events is slow, smarter algorithms have advantage over simple random walk.

Abstract: Concolic testing is an automatic testing technique which combines concrete execution and symbolic execution together.Concolic testing only focuses on programs under test,and lacks the prior knowledge of target faulty statements.From the perspective of detecting defects,time is wasted in generating and executing test inputs that cannot find defects.Furthermore,the limitation of time,budgets,and resources in practical test tasks make this problem even worse.To address this problem,this paper proposes a target-directed testing approach to combine static analysis with concolic testing techniques: the program under test is statically analyzed before testing to identify types and positions of suspicious defects;then,the program is tested with the guidance of the static analysis results.The target-directed testing technique is consisted by 3 steps: Firstly,calculate the reachability relationships from branches to the suspicious statements;Secondly,instrument the program under test for concolic testing;Thirdly,use static analysis information and reachability relationships to guide concolic testing,only generate test inputs can cover suspicious statements,in order to avoid generating test inputs that cannot detect defects.A prototype tool,TARGET,has been implemented based on the proposed technique to test buffer overflows.The authors have experimentally evaluated TARGET on a set of C benchmarks,and the results show that TARGET can find more defects with less time overhead than original concolic testing techniques.

Abstract: Web service testing can be carried out by developers,providers,integrators,third-party certifiers and users,and different testers have different testing objects,testing resources and testing techniques.In realities,testing Web services in the view of users is more practical than others,and can improve testing efficiency in the case study.The authors set up user requirement model firstly,which began with general and abstract object requirements,and then decomposed and refined into concrete and determined atomic Web services,together with such key elements as layer relationships,control structures and restrict conditions,presented as a tree-style model;Next,test case generation and selection is carried out for the object level,service combination level and atomic service level of the user requirement model,and the corresponding techniques are based on equivalence division,data flow analysis and mutation testing;Then,the regression testing would have better efficiency.Based on the dependence relationships in the user requirement model and the fluctuation analysis method,the related paths and variables of Services under testing could be determined quickly;Finally,the workflow and case study are shown with the travel scheduling example.

Abstract: Modeling and testing for a parallel software system is very difficult, because parallel behaviors leads to system state explosion Many testing methods are testing with a large number of redundancies In this paper, a test sequence selection method for parallel software systems is shown The method is based on linear behavior sequences which is preparing for testing, called LBSPT Firstly, a formal model for software system specification is constructed based on Coloured Petri Net (CPN), called system model, and LBSPT is also modeled based on CPN, called LBSPT model A test sequence selection method is defined on these models The method could select all of test sequences related to LBSPT, and does not contain any redundancy The state explode problem in software testing is solved to a certain extent Practices show that the effect of this method is better than other methods such as random walk

Abstract: The present invention discloses an embedded testing module and testing method thereof which encodes one or more test commands to reduce the storage space required by a testing memory. In addition, most functions of automatic test equipment can be replaced by the present invention, in which, through the testing memory according to the present invention, if errors are found during testing, the error information will be transmitted to the external automatic test equipment and the error information can be optionally recorded in a memory. A test operator can get detailed descriptions from the error information stored in the memory, so the test operator can save time for subsequent debugging and tracking operations concerning the errors.

Abstract: Modern diagnosis algorithms are able to identify the defective circuit structure directly from existing fail data without being limited to any specialized fault models. Such algorithms however require test patterns with a high defect coverage, posing a major challenge particularly for embedded testing. In mixed-mode embedded test, a large amount of pseudo-random(PR) patterns are applied prior to deterministic test pattern. Partial Pseudo-Exhaustive Testing (P-PET)replaces these pseudo-random patterns during embedded testing by partial pseudo-exhaustive patterns to test a large portion of a circuit fault-model independently. The overall defect coverage is optimized compared to random testing or deterministic tests using the stuck-at fault model while maintaining a comparable hardware overhead and the same test application time. This work for the first time combines P-PET with a fault model independent diagnosis algorithm and shows that arbitrary defects can be diagnosed on average much more precisely than with standard embedded testing. The results are compared to random pattern testing and deterministic testing targeting stuck-at faults.

Abstract: This thesis aims to explore the viability of using Adaptive Random Testing and Parameterized Random Testing for functional testing of a trading system In particular we will test a subset of the input domain that constitutes sending an order to the exchange We use parameterized Random Testing and Adaptive Random Testing to generate input, and we use both a heuristic oracle and an exception based oracle to verify the output of the trading system Our Random Testing efforts found faults that had previously remained undetected Hence Random Testing has empirically been shown to be a cost efficient supple mentary testing method Furthermore our results indicate that Adaptive Random Testing as good as or better than Random Testing at discovering faults

Abstract: Common problems of high reliability computing are, on on e hand, the magnitude of total testing time required, particularly in the case of high reliability components and, on the other hand, the number of devices under test. In both cases, the objective is to minimize the costs involved in testing without reducing the quality of the data obtained. One solution is based on accelerated life testing techniques which permit to decrease the testing time. Another solution is to incorporate prior beliefs, engineering experience, or previous data into the testing framework. It is in this spirit that the use of a Bayesian approach can, in many cases, significantly reduce the amount of devices required. This paper focuses on a method of building the prior distribution using the previously acquired field data information in such a manner that the newly obtained law is a pseudo-conjugated form of the reliability law describing the product under test.

Abstract: The main purpose of software testing is found a error and then correct it. Random testing selects test cases randomly but it does not explore the previous information. Anti-random testing in which each test applied its total distance from all previous tests is maximum. Anti-Random testing is a variation of random testing, which is the process of generating random input and sending that input to a system for test. In which use hamming Distance and Cartesian Distance for measure of difference.

Abstract: Random testing represents a simple and tractable way for software assessment. This paper presents the Seed tool that can be used for the uniform random generation of recursive data structures such as labelled trees and logical formulas. We show how Seed can be used in several testing contexts, from model based testing to performance testing. Generated data structures are defined by grammar-like rules, given in an XML format, multiplying Seed possible applications. Seed is based on combinatorial techniques, and can generate uniformly at random k structures of size n with an efficient time complexity. Finally, Seed is available as a free Java application and a great effort has been made to make it easy-to-use. Keywords-Random testing; grammar based testing; tool

Abstract: This paper presents a case study of the functional verification of a custom implementation of a random access priority queue which was optimized for performance. Although data structures have been used for decades few studies have examined the effectiveness of different testing strategies applied to complex data structures. In this study, four different testing approaches were used to test a priority queue. The results showed that a state transition testing approach (13 faults discovered) was clearly superior with regards to the number of faults found than the alternatives of a manual testing approach (3 faults discovered), a unit testing approach (4 faults discovered), and a classical test harness approach (6 faults discovered). Because the state transition testing approach used was in essence a modified form of random input testing, the results of this study suggest that the notion that random input testing is typically less effective than other forms of testing may be an overly broad generalization.

Abstract: Recently automated test generation tools that implement variations of symbolic execution technology have demonstrated their ability to find subtle errors. However, one challenge they all face is how to effectively handle the exponential number of paths in large, realistic programs. This paper presents Directed Systematic Dynamic Testing approach, or DSDT for short, that combines compositional systematic dynamic testing with a search heuristic based on block-coverage. The approach tests program under test compositionally and directs the path exploration along the program point with the lowest block-coverage. We implemented our algorithm on top of jFuzz for testing Java program and evaluated our approach on testing a Java implementation of red-black tree. The preliminary experimental results show that Directed Systematic Dynamic Testing approach can efficiently achieve the code coverage of program under test.

Abstract: Commonly, two main classes of formal testing techniques are applied to check the conformance of protocols and software: active and passive testing (monitoring) techniques. Because of many industrial constraints such as the necessity to achieve implementation black box testing, passive testing techniques become highly relevant in several cases. Besides, most of the passive testing techniques only consider control portions of exchanged packets and ignores data parts. However, since systems become more complex, testing for data constraints and relations between packets is now essential. In this paper we propose a novel approach for defining data oriented invariants and show how they can be tested in a trace set by using logic programming. Experimental results on SIP protocol traces are provided.

Abstract: The utility model relates to a low-voltage apparatus multiple-factor reliability testing device, which comprises a central controller, a temperature sensor, a motion frequency detecting circuit, a current ratio detecting circuit and a displayer. The temperature sensor is connected with the central controller and used for detecting working temperature of a low-voltage apparatus, the motion frequency detecting circuit is used for detecting motion frequency of the low-voltage apparatus, and the current ratio detecting circuit is used for detecting ratios between a working load current and a rated load current of the low-voltage apparatus. Reliability testing time of the low-voltage apparatus is relatively long, factors which affect the reliability of the low-voltage apparatus is relatively many. Therefore, if completely randomized multiple-factor tests are conducted, even tests of each level is conducted once; total frequency, time and paid cost may reach a degree which is unbearable. The low-voltage apparatus multiple-factor reliability testing device adopts an orthogonal testing method principle of management to design and analyze methods, thereby greatly reducing frequency and cost of the tests and being extremely appropriate for investigation of the tests like the low-voltage apparatus which is affected by multiple factors.

Abstract: There are many problems in the third-party testing of complex software,such that the occurrence of software failure is a very complicated process with uncertainty;it is difficult for tester to select representative test cases;the cost and resource for testing are very limited;the process of testing is inefficient.A third-party adaptive testing approach for complex software is presented by the test cases similarity measure from two levels and testing process is regarded as the implementation of state sequence according to the practice of software engineering and the correlated feature of software failures.Finally this approach is applied in the practical testing.

Abstract: In an environment where software development is expanding in scope, it is necessary to develop new techniques that can be used to identify defects in a product, while at the same time keeping the cost of product certification to a minimum. For the initial deployment of any software, there is no way to get around performing a complete test of the product. However, subsequent iterations of software development require complete testing of the new features, and regression-testing of the existing features using orthogonal defect classification. Random sampling techniques may also be used; but unless they are properly designed, run the risk of not being adequately representative of the whole array of test cases. Using orthogonal arrays, we can break the test cases down into categories, thereby reducing the amount of testing without sacrificing the quality of the deliverables.

Abstract: In order to perform the dynamic testing of the white-box and black box,the dynamic testing tool based on source code instrumentation pile is designed.This tool contains the preprocessing methods of source code,the design of instrumentation library,instrumentation strategy and statistical analysis.The high accuracy can be obtained through the syntax and lexical analyses of the source code.The instrumentation information is written into a file before the fulfilment of the function,the way which can reduce a large number of IO operations.The testing parameters of code coverage rate,execution duration and complexity were obtained.

Abstract: Search-based Software Engineering has been utilized for a number of software engineering activities. One area where Search-Based Software Engineering has seen much application is test data generation. Evolutionary testing designates the use of metaheuristic search methods for test case generation. The search space is the input domain of the test object, with each individual or potential solution, being an encoded set of inputs to that test object. The fitness function is tailored to find test data for the type of test that is being undertaken. Evolutionary Testing (ET) uses optimizing search techniques such as evolutionary algorithms to generate test data. The effectiveness of GA-based testing system is compared with a Random testing system. For simple programs both testing systems work fine, but as the complexity of the program or the complexity of input domain grows, GA-based testing system significantly outperforms Random testing.