Abstract: We report the results of three empirical studies of fault detection and stability performance of the predicate-based BOR (Boolean Operator) testing strategy. BOR testing is used to develop test cases based on formal software specification, or based on the implementation code. We evaluated the BOR strategy with respect to some other strategies by using Boolean expressions and actual software. We applied it to software specification cause-effect graphs of a safety-related real-time control system, and to a set of N-version programs. We found that BOR testing is very effective at detecting faults in predicates, and that BOR-based approach has consistently better fault detection performance than branch testing, thorough (but informal) functional testing, simple state-based testing, and random testing. Our results indicate that BOR test selection strategy is practical and effective for detection of faulty predicates and is suitable for generation of safety-sensitive test-cases. >

Abstract: In this paper, we report the results of four empirical studies for evaluating a predicate-based software testing strategy, called BOR (Boolean operator) testing. The BOR testing strategy focuses on the detection of Boolean operator faults in a predicate, including incorrect AND/OR operators and missing or extra NOT operators. Our empirical studies involved comparisons of BOR testing with several other predicate-based testing strategies, using Boolean expressions, a real-time control system, and a set of N-version programs. For program-based test generation, BOR testing was applied to predicates in a program. For specification-based test generation, BOR testing was applied to cause-effect graphs representing software specification. The results of our studies indicate that BOR testing is practical and effective for both specification- and program-based test generation.

Abstract: This article describes an easy-to-use program for computer-adaptive testing. Two files have to be provided to use the program: the bank of items and the file containing their parameters. The program has been checked by simulation. Data on the accuracy of ability estimation are offered. Details on how to run the program and the format of both required files are also provided.

Abstract: Statistical testing is based on a probabilistic generation of test data: classical structural or functional criteria serve as guides for defining an input profile and a test size. The method is intended to compensate for the imperfect connection of current criteria with software faults, and should not be confused with random testing, a "blind" approach that uses a uniform profile over the input domain. This paper reports on experimental results obtained on a software component from the nuclear field: • unit testing of four functions ‐ statistical input sets were designed according to structural criteria; their efficiency was compared to the one of 1) deterministic sets derived from the same criteria and 2) uniform random sets; the comparison involved 2816 faults of mutation type seeded one by one in the source codes. • whole component testing ‐ statistical functional testing was designed from behaviour models of the component: finite state machines, decision tables, Statecharts; its efficiency was compared to the one of random testing, using two versions of the component: the real one, in which a minor fault was found, and a student version with 12 revealed faults. The results showed the high fault revealing power of statistical testing, and its best efficiency in comparison to deterministic and random testing.

Abstract: This paper describes the optimal testing input sets required for fault diagnosis of nuclear power plant digital electronic circuits. With complicated systems such as very large scale integration (VLSI), nuclear power plant (NPP), and aircraft, testing is the major factor of the maintenance of the system. Particularly, the diagnosis time grows quickly with the complexity of the component. For a reduced diagnosis time we derived the optimal testing sets that are the minimal testing sets required for detecting the failure and for locating the failed component. For a reduced diagnosis time, the technique presented by Hayes (1971) fits best for our approach to testing set generation among many conventional methods. However, this method has the following disadvantages: a) it considers only a simple network; b) it concerns only whether the system is in a failed state or not and does not provide the way to locate the failed component. Therefore we have derived the optimal testing input sets that resolve these problems by Hayes while preserving its advantages. When we applied the optimal testing sets to the automatic fault diagnosis system (AFDS) which incorporates the advanced fault diagnosis method of the artificial intelligence technique, we found that fault diagnosis using the optimal testing sets makes testing of the digital electronic circuits much faster than that using exhaustive testing input sets; when we applied them to test the Universal (UV) Card which is a nuclear power plant digital input/output solid state protection system card, we reduced the testing time by up to about 100. >

Abstract: Statistical Process Control testing of manufacturing processes began back in the 1940's with the development of Process Control Charts by Dr. Waiter A. Shewart. Sandia National Laboratories has developed an application of the SPC method for performance testing of electronic security equipment. This paper documents the evaluation of this testing methodology applied to electronic security equipment and an associated laptop computer-based system for obtaining and analyzing the test data. Sandia developed this SPC sensor performance testing method primarily for use on portal metal detectors, but, has evaluated it for testing of an exterior intrusion detection sensor and other electronic security devices. This method is an alternative to the traditional binomial (alarm or no-alarm) performance testing. The limited amount of information in binomial data drives the number of tests necessary to meet regulatory requirements to unnecessarily high levels. For example, a requirement of a 0.85 probability of detection with a 90% confidence requires a minimum of 19 alarms out of 19 trials. By extracting and analyzing measurement (variables) data whenever possible instead of the more typical binomial data, the user becomes more informed about equipment health with fewer tests (as low as five per periodic evaluation). >

Abstract: The basic unit of testing in an object-oriented program is a class. Although there has been much recent research on testing of classes, most of this work has focused on black-box approaches. However, since black-box testing techniques may not provide sufficient code coverage, they should be augmented with code-based or white-box techniques. Dataflow testing is a code-based testing technique that uses the dataflow relations in a program to guide the selection of tests. Existing dataflow testing techniques can be applied both to individual methods in a class and to methods in a class that interact through messages, but these techniques do not consider the dataflow interactions that arise when users of a class invoke sequences of methods in an arbitrary order. We present a new approach to class testing that supports dataflow testing for dataflow interactions in a class. For individual methods in a class, and methods that send messages to other methods in a the class, our technique is similar to existing dataflow testing techniques. For methods that are accessible outside the class, and can be called in any order by users of the class, we compute dataflow information, and use it to test possible interactions between these methods. The main benefit of our approach is that it facilitates dataflow testing for an entire class. By supporting dataflow testing of classes, we provide opportunities to find errors in classes that may not be uncovered by black-box testing. Our technique is also useful for determining which sequences of methods should be executed to test a class, even in the absence of a specification. Finally, as with other code-based testing techniques, a large portion of our technique can be automated.