Towards behavior-driven graphical user interface testing

TL;DR: The specification language Slang introduced by this paper generates automatically executable test cases from BDD-like feature descriptions that integrate low-fidelity prototypes in form of wireframesketcher models using Slang to address behavior-driven test case design challenges.

Abstract: The majority of users interacts with an application through its graphical user interface (GUI). To ensure high quality and expected behavior, those graphical user interfaces have to be tested thoroughly. Yet, creating graphical user interface test cases is considered expensive in comparison to unit or integration tests. In addition, test cases are perceived to be expensive to run and brittle, therefore causing a lot of false negative test results. Behavior-driven test case design addresses this challenges by bringing requirement specifications and test cases closer together. Although industry-proven tools map test specifications automatically, test methods making test scripts executable need to be implemented manually. The specification language Slang introduced by this paper generates automatically executable test cases from BDD-like feature descriptions that integrate low-fidelity prototypes in form of wireframesketcher models. To quantify the economic advantage of our approach an AB/BA crossover designed experiment was conducted. The experiment showed that creating automatically executable test cases utilizing Slang takes 63% less time compared to the industry-proven tool JBehave. In addition to presenting the experiment's results, the paper elaborates on first experience from applying the approach in a large Swiss bank. The findings of our experiments are supported by results from applying our approach in real-world scenarios. In addition, experiment as well as case study participants appreciated the sophisticated editor support of Slang.