Abstract: Adaptive random testing (ART) describes a family of algorithms for generating random test cases that have been experimentally demonstrated to have greater fault-detection capacity than simple random testing. We outline and demonstrate two new ART algorithms, and demonstrate experimentally that they offer similar performance advantages, with considerably lower overhead than other ART algorithms.

Abstract: The F-measure - the number of distinct test cases to detect the first program failure - is an effectiveness measure for debug testing strategies. We show that for random testing with replacement, the F-measure is distributed according to the geometric distribution. A simulation study examines the distribution of two adaptive random testing methods, to study how closely their sampling distributions approximate the geometric distribution, revealing that in the worst case scenario, the sampling distribution for adaptive random testing is very similar to random testing. Our results have provided an answer to a conjecture that adaptive random testing is always a more effective alternative to random testing, with reference to the F-measure. We consider the implications of our findings for previous studies conducted in the area, and make recommendations to future studies.

Abstract: The goal of testing is to gain confidence in a physical computer based system by means of executing it. More than one third of typical project resources are spent on testing embedded and real-time systems, but still it remains ad-hoc, based on heuristics, and error-prone. Therefore systematic, theoretically well-founded and effective automated real-time testing techniques are of great practical value. Testing conceptually consists of three activities: test case generation, test case execution and verdict assignment. We present T-UPPAAL-a new tool for model based testing of embedded real-time systems that automatically generates and executes tests "online" from a state machine model of the implementation under test (IUT) and its assumed environment which combined specify the required and allowed observable (realtime) behavior of the IUT. T-UPPAAL implements a sound and complete randomized testing algorithm, and uses a formally defined notion of correctness (relativized timed input/output conformance) to assign verdicts. Using online testing, events are generated and simultaneously executed.

Abstract: One of the strengths of using stream X-machines to specify a system is that, under certain well defined conditions, it is possible to produce a test set that is guaranteed to determine the correctness of an implementation. However, the existing method assumes that the implementation of each processing function is proved to be correct before the actual testing can take place, so it only test the system integration. This paper presents a new method for generating test sets from a deterministic stream X-machine specification that generalises the existing integration testing method. This method no longer requires the implementations of the processing functions to be proved correct prior to the actual testing. Instead, the testing of the processing functions is performed along with the integration testing.

Abstract: Software Model-Checking and Testing are some of the most used techniques to analyze software systems and identify hidden faults. While software model-checking allows for an exhaustive and automatic analysis of the system expressed through a model, software testing is based on a clever selection of “relevant” test cases, which may be manually or automatically run over the system.

Abstract: We present a method for optimizing software testing efficiency by identifying the most error prone path clusters in a program. We do this by developing variable length genetic algorithms that optimize and select the software path clusters which are weighted with sources of error indexes. Although various methods have been applied to detecting and reducing errors in a whole system, there is little research into partitioning a system into smaller error prone domains for testing. Exhaustive software testing is rarely possible because it becomes intractable for even medium sized software. Typically only parts of a program can be tested, but these parts are not necessarily the most error prone. Therefore, we are developing a more selective approach to testing by focusing on those parts that are most likely to contain faults, so that the most error prone paths can be tested first. By identifying the most error prone paths, the testing efficiency can be increased.

Abstract: Testing is a necessary, but costly process for user-centric quality control. Moreover, testing is not comprehensive enough to completely detect faults. Many formal meth ods have been pro posed to avoid the drawbacks of testing, e.g., model checking that can be automatically carried out. This paper presents an approach that (i) generates test cases from the specification and (ii) transfers the specification-oriented testing process to model checking. Thus, the approach combines the advantages of testing and model checking assuming the availability of (i) a model that specifies the ex pected, desirable system behavior as required by the user and (ii) a second model that describes the system behavior as observed. The first model is complemented in also specifying the undesirable system properties. The approach analyzes both these specification models to generate test cases that are then converted into temporal logic formulae to be model checked on the second model.

Abstract: In this paper, we propose a testing technique that can directly test FBD programs without generating intermediate code for testing purpose. The previous PLC-based software testing generates an intermediate code such as C, which is equivalent to the original FBD, and targets an intermediate code. In order to apply unit and integration testing techniques to FBDs, we transform FBD program into a control flow graph and apply existing control flow testing coverage criteria to the graph. With our approach, PLC based software designed in FBD language can be tested cost-efficiently because we do not need to generate intermediate code. To demonstrate the usefulness of the proposed method, we use a trip logic of BP(Bistable Process) in DPPS(Digital Plant Protection System) RPS(Reactor Protection System), which is currently being developed at KNICS (KNICS, -) in Korea.

Abstract: Component-based software development has posed a serious challenge to system verification since externally-obtained components could be a new source of system failures. This issue can not be completely solved by either model-checking or traditional software testing techniques alone due to several reasons: 1) externally obtained components are usually unspecified/partially specified; 2)it is generally difficult to establish an adequacy criteria for testing a component; 3)components may be used to dynamically upgrade a system. This paper introduces a new approach (called {\em model-checking driven black-box testing}) that combines model-checking with traditional black-box software testing to tackle the problem in a complete, sound, and automatic way. The idea is to, with respect to some requirement (expressed in CTL or LTL) about the system, use model-checking techniques to derive a condition (expressed in communication graphs) for an unspecified component such that the system satisfies the requirement iff the condition is satisfied by the component, and which can be established by testing the component with test cases generated from the condition on-the-fly. In this paper, we present model-checking driven black-box testing algorithms to handle both CTL and LTL requirements. We also illustrate the idea through some examples.

Abstract: A new methodology is presented to allocate testing units to the different components within a system when the system configuration is fixed and there are budgetary constraints limiting the amount of testing. The objective is to allocate additional testing units so that the variance of the system reliability estimate, at the conclusion of testing, will be minimized. Testing at the component-level decreases the variance of the component reliability estimate, which then decreases the system reliability estimate variance. The difficulty is to decide which components to test given the system-level implications of component reliability estimation. The results are enlightening because the components that most directly affect the system reliability estimation variance are often not those components with the highest initial uncertainty. The approach presented here can be applied to any system structure that can be decomposed into a series-parallel or parallel-series system with independent component reliability estimates. It is demonstrated using a series-parallel system as an example. The planned testing is to be allocated and conducted iteratively in distinct sequential testing runs so that the component and system reliability estimates improve as the overall testing progresses. For each run, a nonlinear programming problem must be solved based on the results of all previous runs. The testing allocation process is demonstrated on two examples.

Abstract: The use of a model to describe and test the expected behavior of a program is a well-proved software testing technique Statistical structural testing aims at building a model from which an input probability distribution can be derived that maximizes the coverage of some structural criteria by a random test data generator Our approach consists in converting statistical structural testing into a probabilistic concurrent constraint programming (PCCP) problem in order 1) to exploit the high declarativity of the probabilistic choice operators of this paradigm and 2) to benefit from its automated constraint solving capacity This paper reports on an ongoing work to implement PCCP and exploit it to solve instances of statistical structural testing problems Application to testing Java Card applets is discussed

Abstract: This paper presents a network graph model for test case automatic generation, which is based on the combinatorial coverage of all the interface parameters for black box testing. In this model, a path from left to right represents a test case. The algorithm chooses a test case every time to make the corresponding path in the network graph cover all the uncovered vertices to the greatest degree by some rules. It can generate a test case table to cover all the pair wise combination of all the interface parameters. Contrasted with the existing work, authors prove that the generated test cases are able to cover all the combinations of parameters to the greatest degree with the smallest scale of test suite, thus it can improve the quality of software testing by decreasing software testing cost and improving its efficiency.

Abstract: The invention involves a semi-conductor component testing process, and a system for testing semi-conductor components, in which a central computer device, in particular a central test apparatus is provided, with which test result data obtained from at least two separate tests is jointly evaluated, in particular by means of an appropriate pattern recognition process, which incorporates the test result data obtained from the separate tests into the analysis.

Abstract: It is necessary to assess the reliability of distributed safety-critical systems to a high degree of confidence before they are deployed in the field. However, distributed safety-critical software systems often include some rarely executed critical functions that are often inadequately tested in statistical testing based reliability estimation. This paper presents a method that can accelerate statistical testing of distributed safety-critical software. The method starts with the derivation of scenario usage diagram model (SUD) from UML diagrams annotated with usage related attributes and reliability attributes. Then the statistical testing accelerating method based on importance sampling is presented. When both the critical scenarios and the entire software are adequately tested, the method can still compute the unbiased software reliability from the test results with much less test cases. Thus, the statistical testing cost of distributed safety-critical software can be reduced effectively.

Abstract: With the continuous increase in design complexities and packing densities of integrated circuit (IC), problems associated with conventional Automatic Test Equipment (ATE)-based IC testing approach have become a burning issue in the semiconductor world, which needs an economic solution with reliable performance. Recently, the superiority of Dynamic Reseeding-based Mixed-mode (DRM) technique has been proven over all other existing test techniques in the Built-in Self-Test (BIST) environment. This thesis introduces the implementation of the DRM technique in a system-on-a-chip (SOC) in alleviating the problems of conventional ATE-based external testing of digital IC. The performance of the SOC in IC testing has been verified using fault simulation experiments on the ISCAS85 benchmark circuits (Circuits proposed in the International Symposium on Circuits and Systems in 1985). Significant improvement is observed in achieving complete fault coverage for the ISCAS85 benchmark circuits using acceptable number of test vectors. Fault simulation results show that the proposed DRM technique produces 100% fault coverage for the benchmark circuits c432, c1355, c1908, c2670, c3540 and c5315 using the 232, 526, 996, 336, 360 and 748 test cubes, respectively which are much lower than the numbers from the approaches suggested by other researchers. It also offers much lower data storage requirements in IC testing than the conventional ATE-based testing approach. The results show that 2 to 11 times less memory is needed for testing the ISCAS85 benchmark circuits using the DRM technique than that of the deterministic testing approach. Verilog Hardware Description Language (HDL), which is an industry standard IC design tool, has been used to design the SOC proposed in this thesis. Main modules of the SOC are micro-UART (Universal Asynchronous Receiver and Transmitter), a controller, pattern generator, signature analyzer (SA), instruction registers and Random Access Memories (RAMs). A prototype test set-up has been developed for testing IC by implementing the design of the SOC into a Field Programmable Gate Array Logic (FPGA) chip and then by interfacing the FPGA chip with a personal computer (PC) through a Graphical User Interface (GUI). For testing a circuit, necessary test information is loaded into the SOC and the testing process is executed using the GUI from the PC. The SOC goes into autonomous mode. It generates test vectors, applies them to the Circuit Under Test (CUT) and captures the output responses and sends it into the SA for compression. At the end of testing, the generated signature is compared with that of a reference circuit (fault-free circuit of the same type) and the CUT is identified as fault-free if the two signatures are the same and as faulty if otherwise. The operation of the SOC has been verified in real time by testing a 16-bit multiplier as a sample CUT. It is user programmable, which increases flexibility and reliability in IC testing. It is capable of testing functionality of combinational circuits as well as sequential circuits with scan-path facility.

Abstract: SoCs are in general built with embedded IP cores, each of which is procured from different IP providers with no prior information on known-good-yield (KGY). In practice, partial testing is a practical choice for assuring the yield of the product under the stringent time-to-market requirements. Therefore, a proper sampling technique is a key to high confidence testing and cost effectiveness. Based on previous research, this paper proposes a novel statistical testing technique for increasingly hybrid integrated systems fabricated on a single silicon die with no a-priori empirical yield data. This problem is referred to as the unknown-good-yield (UKGY) problem. The proposed testing method, referred to as regressive testing (RegT) in this paper, exploits another way around by using parameters (referred to as assistant variables (AV)) that are employed to evaluate the yields of randomly sampled SoCs and thereby estimating the good yield by using a regression analysis method with regard to confidence intervals. Numerous simulations are conducted to demonstrate the efficiency and effectiveness of the proposed RegT in comparison to characterization-based testing methods.

Abstract: A method, system and computer program product for generating a coverage model to describe a testing scheme for a simulated system are disclosed. In a preferred embodiment, a simulated system is tested with a testing simulation program. A simple event database is generated with the testing simulation program. Results of a checker analysis from the testing with the testing simulation program are obtained, and coverage data is created from a coverage model configuration file, the simple event database and the results of the checker analysis.

Abstract: The paper analyzes the disadvantages of the conventional testing system for the single-phase AC series-excited motor (SACSM), introduces a kind of no-coupling motor testing system: the motor testing without the sensor speed detection; constructs the mathematic model and gives the method of parameter identification; describes the method of speed detection; puts forward a kind of new genetic algorithm based on the social hierarchic rank which is used to parameter identification of motor model in SACSM testing system, and adopts the idea that reduces searching scope of genes according to individual fitness, and designs a changeable scope searching operator which makes the genetic algorithm optimize parameters of motor more effectively. Experiments show that the testing system can be maintained conveniently to help obtain the desired results.