A practical model-based statistical approach for generating functional test cases: application in the automotive industry

TL;DR: A model‐based statistical testing approach that automatically generates online and offline test cases for embedded software is developed and significant improvements in performing functional unit testing in a real and complex industrial context are obtained.

Abstract: With the growing complexity of industrial software applications, industrials are looking for efficient and practical methods to validate the software. This paper develops a model-based statistical testing approach that automatically generates online and offline test cases for embedded software. It discusses an integrated framework that combines solutions for three major software testing research questions: i how to select test inputs; ii how to predict the expected results of a test; and iii when to stop testing software. The automatic selection of test inputs is based on a stochastic test model that accounts for the main particularity of embedded software: time sensitivity. Software test practitioners may design one or more test models when they generate random, user-oriented, or fault-oriented test inputs. A formal framework integrating existing and appropriate specification techniques was developed for the design of automated test oracles executable software specifications and the formal measurement of functional coverage. The decision to stop testing software is based on both test coverage objectives and cost constraints. This approach was tested on two representative case studies from the automotive industry. The experiment was performed at unit testing level in a simulated environment on a host personal computer automatic test execution. The two software functionalities tested had previously been unit tested and validated using the test design approach conventionally used in the industry. Applying the proposed model-based statistical testing approach to these two case studies, we obtained significant improvements in performing functional unit testing in a real and complex industrial context: more bugs were detected earlier and in a shorter time. Copyright © 2012 John Wiley & Sons, Ltd.