Automating functional tests using Selenium

TL;DR: The standard environment for testing with Selenium is described, as well as modifications the authors performed to incorporate their script pages into a wiki, and whether additional automated functional testing below the GUI layer was still necessary and/or appropriate.

Abstract: Ever in search of a silver bullet for automated functional testing for Web applications, many folks have turned to Selenium. Selenium is an open-source project for in-browser testing, originally developed by ThoughtWorks and now boasting an active community of developers and users. One of Selenium's stated goals is to become the de facto open-source replacement for proprietary tools such as WinRunner. Of particular interest to the agile community is that it offers the possibility of test-first design of Web applications, red-green signals for customer acceptance tests, and an automated regression test bed for the Web tier. This experience report describes the standard environment for testing with Selenium, as well as modifications we performed to incorporate our script pages into a wiki. It includes lessons we learned about continuous integration, script writing, and using the Selenium Recorder (renamed IDE). We also discuss how long it took to write and maintain the scripts in the iterative development environment, how close we came to covering all of the functional requirements with tests, how often the tests should be (and were) run, and whether additional automated functional testing below the GUI layer was still necessary and/or appropriate. While no silver bullet, Selenium has become a valuable addition to our agile testing toolkit, and is used on the majority of our Web application projects. It promises to become even more valuable as it gains widespread adoption and continues to be actively developed.