IP-XACT based system level mutation testing

TL;DR: From the results, it can be seen that the defined IP-XACT mutation serves an effective qualification for simulation tests, in terms of its ability to reveal the weakness of the tests.

Abstract: Mutation-testing has been considered as an important coverage metric to measure the quality of simulation-based verification and validation processes [1, 2, 3]. On the other hand, IP-XACT has evolved to the IEEE standard for IP reuse and IP-based System-on-Chip (SoC) integration, which covers both RTL and TLM. In this paper, we present our effort to enable the mutation-based simulation coverage metric for system level IP integration with IP-XACT. Two major ingredients are required for this extension. First, as IP-XACT system designs are XML files, which are not originally for execution, we need an execution/simulation engine for IP-XACT designs. For this, we created a code generator that generates SystemC models from IP-XACT XML designs, such that we can simulate and test an IP-XACT design. Second, we define the mutation operators on IP-XACT schema, which is the model of errors that we can inject into IP-XACT designs during mutation testing. With IP-XACT, the mutation maintains a focus on the integration and configuration of components. We implemented the code generator and mutation operators in an Eclipsed-based IP-XACT editor with the help of Eclipse Modeling Framework. Then several experiments were conducted on a TLM library for CoreConnect SoC modeling. From the results, we can see that the defined IP-XACT mutation serves an effective qualification for simulation tests, in terms of its ability to reveal the weakness of the tests.