Abstract: This paper models software reliability and testing costs using a new tool: a quasi-renewal process. It is assumed that the cost of fixing a fault during software testing phase, consists of both deterministic and incremental random parts, increases as the number of faults removed increases. Several software reliability and cost models by means of quasi-renewal processes are derived in which successive error-free times are independent and increasing by a fraction. The maximum likelihood estimates of parameters associated with these models are provided. Based on the valuable properties of quasi-renewal processes, the expected software testing and debugging cost, number of remaining faults in the software, and mean error-free time after testing are obtained. A class of related optimization problem is then contemplated and optimum testing policies incorporating both reliability and cost measures are investigated. Finally, numerical examples are presented through a set of real testing data to illustrate the models results.

Abstract: SUMMARY A strategy described as ‘testing using M model programs’ (abbreviated to ‘M-mp testing’) is investigated as a practical alternative to software testing based on manual outcome prediction. A model program implements suitably selected parts of the functional specification of the software to be tested. The M-mp testing strategy requires that M (M ≥ 1) model programs as well as the program under test, P , should be independently developed. P and the M model programs are then subjected to the same test data. Difference analysis is conducted on the outputs and appropriate corrective action is taken. P and the M model programs jointly constitute an approximate test oracle. Both M-mp testing and manual outcome prediction are subject to the possibility of correlated failure. In general, the suitability of M-mp testing in a given context will depend on whether building and maintaining model programs is likely to be more cost effective than manually pre-calculating P ’s expected outcomes for given test data. In many contexts, M-mp testing could also facilitate the attainment of higher test adequacy levels than would be possible with manual outcome prediction. A rigorous experiment in an industrial context is described in whichM-mp testing (withM = 1) was used to test algorithmically complex scheduling software. In this case, M-mp testing turned out to be significantly more cost effective than testing based on manual outcome prediction. Copyright  2001 John Wiley & Sons, Ltd.

Abstract: At the class testing level, state-based testing and data flow testing techniques have been employed. However, while the former only involves the variables that have an effect on the behaviour of the object under test, it is possible for errors to occur in variables, which do not define an object’s state. Data flow testing has been applied to generate test cases for testing classes using data flow criteria, but this is a difficult task. Moreover, some of data flow test cases thus generated may be unworkable. Selecting data flow test cases based on sequences of specification messages is a way to reduce the effort of generating feasible intra-class data flow test cases. However, some test cases cannot be selected, if data flow anomalies exist within the sequences. The data flow testing technique in this research is divided into two stages; first detecting data flow anomalies and then computing data flow test cases.

Abstract: From a statistical point of view, adaptive testing belongs to the class of problems of selecting a best subset of random variables. As in any other instance of this problem, adaptive testing is likely to capitalize on estimation error. An empirical example of the phenomenon is discussed, and a strategy for cross-validating results from adaptive testing is proposed. Results from a simulation study on the strategy are presented.

Abstract: A central activity in software testing is the problem of finding program faults. The observation-based testing methodology can be applied to this activity. Experimental results show that the use of cluster filtering to selectively identify test cases in the observation-based testing methodology is a reliable and efficient technique for the identification of failures in a population of program executions generated during late stage system testing. A traditional method for identifying program faults is partition testing. Partition testing uses test cases derived from functional or structural subdomains. The method suffers from difficulty in forming clusters, the lack of correlation of the subdomains with faults, and the fact that the faults found may not be representative of the failures commonly seen by users. Observation-based testing is an alternative methodology for software testing. It collects a population of runs from actual program usage. Any failures in this population will be representative of failures seen by the customer. The cluster filtering technique uses cluster analysis to partition the population based on the runs' execution profiles. It employs a sampling method to select test cases from that partitioning. Observation-based testing has been shown to be a viable method for efficiently estimating program reliability. This work applies the method to debug testing. A series of experiments were undertaken to evaluate its utility. The results of those experiments show that (1) failed executions are concentrated in an identifiable clusters, namely those of small size; (2) the probability of finding any of the failed executions is, on the average, no worse than random sampling and in many cases is guaranteed; (3) the process is substantially more effective and efficient than random sampling at finding failed executions; and (4) the adaptive sampling method takes advantage of the homogeneity of the clusters to find more failed executions.

Abstract: We present novel test set encoding and pattern decompression methods for core-based systems. These are based on the use of twisted-ring counters and offer a number of important advantages-significant test compression (over 10/spl times/ in many cases), less tester memory and reduced testing time, the ability to use a slow tester without compromising test quality or testing time, and no performance degradation for the core render test. Surprisingly, the encoded test sets obtained from partially-specified test sets (test cubes) are often smaller than the compacted test sets generated by automatic test pattern generation programs. Moreover, a large number of patterns are applied test-per-clock to cores, thereby increasing the likelihood of detecting non-modelled faults. Experimental results for the ISCAS benchmark circuits demonstrate that the proposed test architecture offers an attractive solution to the problem of achieving high test quality and low testing time with relatively slower, less expensive testers.

Abstract: In order to achieve reproducible and deterministic functional integration testing of real-time systems software it is essential to consider inputs, outputs, and the order in which the tasks communicate and synchronize with each other. In this paper we present a method for deterministic integration testing of strictly periodic fixed priority scheduled real-time systems. Essentially it is a structural white box testing method applied on the system level rather than the individual programs of the tasks. The method includes a reachability technique for deriving all possible orderings of task starts, preemptions and completions for tasks executing in a system where synchronization is resolved using the Priority Ceiling Emulation Protocol (PCEP) or offsets. The method also includes a structural testing strategy for achieving full coverage with respect to the derived execution orderings. The testing strategy also allow test methods for sequential programs to be applied, since each identified ordering can be regarded as a sequential program. In the presented analysis and testing strategy, we consider task sets with recurring release patterns, and take into account the effects of variations in start and execution times of the involved tasks, as well as the variations of the arrival and duration of the critical sections.

Abstract: We propose an autonomous, deterministic scan-BIST architecture that allows compact, precomputed test sets with complete fault coverage to be used for field testing The use of such short test sequences is desirable in safety-critical systems since it reduces the error latency It also reduces testing time and therefore allows periodic field testing to be carried out with low system downtime We synthesize the BIST logic for several ISCAS 89 benchmarks and industrial circuit modules and show that the BIST overhead is low in all cases The proposed design can also be efficiently used with a mixed-mode BIST strategy

Abstract: Traditionally, software reliability models have required that failure data be gathered using only representative testing methods. Over time, however, representative testing becomes inherently less effective as a means of improving the actual quality of the software under test. Additionally, the use of failure data based on observations made during representative testing has been criticized because of the statistical noise inherent in this type of data. In this dissertation, a testing method is proposed to make reliability testing more efficient and accurate. Representative testing is used early, when the rate of fault revelation is high. Directed testing is used later in testing to take advantage of its faster rate of fault detection. To make use of the test data from this mixed method approach to testing, a software reliability model is developed that permits reliability estimates to be made regardless of the testing method used to gather failure data. The key to being able to combine data from both representative testing and directed testing is shifting the random variable used by the model from observed interfailure times to a postmortem analysis of the debugged faults and using order statistics to combine the observed failure rates of faults no matter how those faults were detected. This shift from interfailure times removes the statistical noise associated with the use of this measure, which should allow models to provide more accurate estimates and predictions. Several experiments were conducted during the course of this research. The results from these experiments show that using the mixed method approach to testing with the new model provides reliability estimates that are at least as good as estimates from existing models under representative testing, while requiring fewer test cases. The results of this work also show that the high level of noise present in failure data based on observed failure times makes it very difficult for models that use this type of data to make accurate reliability estimates. These findings support the suggested move to the use of more stable quantities for reliability estimation and prediction.

Abstract: A new method of built-in self-test of a real time digital signal processor is offered. This method is a hybrid one of board-level testing and system-level testing. It can integrate system adjustment, on-line testing and off-line testing in one BIST architecture. The implementation of the BIST is offered and the performance of the time domain processing test and frequency domain processing test are analysed.

Abstract: Many researchers have compared the fault detecting ability of random testing and partition testing with test cases replacement. Great achievements have been gained. Comparing random testing with partition testing without test cases replacement has not been found in existing documents. However, in practice, especially at the early stage of testing and during module testing, test cases are not replaced. Therefore, it is necessary to compare random with partition testing under this case. In this paper it is done and four findings are proposed. One difference from the results done by Chen and Yu is that when the domain is divided into such subdomains with the same size, the same failures and the same test numbers, partition testing performed worst than random testing. How to form a subdomain with heavy failures by use of all kinds of information and then gives it a large number of tests is essential for partition testing.

Abstract: Pairwise testing is a specification-based testing criterion, which requires that for each pair of input parameters of a system, every combination of valid values of these two parameters be covered by at least one test case. In this paper we propose a new test generation strategy for pairwise testing.

Abstract: An advanced scan array architecture considering testing power reduction is proposed, in which a wrapper and two dimensional scan chain is adopted. Furthermore pseudo-BIST is integrated with the wrapper to cut down the testing time. Experimental results of industrial circuits show that testing power is reduced, observable and is close to the functional power. At the same time, testing time is also cut down deriving benefits from pseudo-BIST.

Abstract: Method for testing channel modules (101) or control modules (102) containing scan chains, with testing using sampling vectors from a test register (103) that provides a control sample module signal (131) for the control module and a channel sample module signal (111) for the channel module. Channel and control modules can be tested a number of times with same scan chain, so memory saving testing is efficient. An Independent claim is made for a testing device with logic software provided for testing sampling output signals.

Abstract: Although testing is the most popular method for assuring software quality, there are two recognized limitations, known as the reliable test set problem and the oracle problem. Fault-based testing is an attempt by Morell to alleviate the reliable test set problem. In this paper, we propose to enhance fault-based testing to address the oracle problem as well. We present an integrated method that combines metamorphic testing with fault-based testing using real and symbolic inputs.

Abstract: Integration testing has always been a challenge especially if the system under test is large with many subsystems and interfaces. This paper proposes an approach to design End-to-End (E2E) integration testing, including test scenario specification, test case generation and tool support. Test scenarios are specified as thin threads, each of which represents a single function from an end user's point of view. Thin threads can be organized hierarchically into a tree with each branch consisting of a set of related thin threads representing a set of related functionality. A test engineer can use thin-thread trees to generate test cases systematically, as well as carry out other related tasks such as risk analysis and assignment, regression testing, ripple effect analysis. A prototype tool has been developed to support E2E testing in a distributed environment on the J2EE platform.

Abstract: Mutation testing is a powerful, but computationally expensive, technique for unit testing software. This expense has prevented mutation form becoming widely used in practical situations, but recent engineering advances have given us techniques and algorithms for significantly reducing the cost of mutation testing. These technique include a new algorithmic execution technique include a new algorithmic execution technique called schema-based mutation, a reduction technique called selective mutation, heuristics for detecting equivalent mutants, and algorithms for automatic test data generation. This paper reviews experimentation with these advances and outlines a design for a system that will approximate mutation, but in a way that will be accessible to every day programmers. We envision a system to which a programmer can submit a program unit and get back a set of input/output pairs that are guaranteed to form an effective test of the unit by being close to mutation adequate. We believe this system could be efficient enough to be adopted by leading-edge software developers. Full automation in unit testing has the potential to dramatically change the economic balance between testing and development, by reducing the cost of testing from the major part of the total development cost to a small fraction.

Abstract: Early studies of random versus partition testing used the probability of detecting at least one failure as a measure of test effectiveness and indicated that partition testing is not significantly more effective than random testing. More recent studies have focused on proportional partition testing because a proportional allocation of the test cases (according to the probabilities of the subdomains) can guarantee that partition testing will perform at least as well as random testing. We show that this goal for partition testing is not a worthwhile one. Guaranteeing that partition testing has at least as high a probability of detecting a failure comes at the expense of decreasing its relative advantage over random testing. We then discuss other problems with previous studies and show that failure to include important factors (cost, relative effectiveness) can lead to misleading results.