Continuous experimentation for software developers

TL;DR: In this article, the authors propose a data-driven approach to manage multiple experiments in parallel (i.e., operating and monitoring multiple versions), and specify parameters for experiments (e.g., to avoid that they negatively impact each other).

Abstract: Recent advances in build, test, and deployment automation not only enable companies shipping new functionality faster to their users, but also provide them the ability to experiment with functionality on small fractions of the user base first. These experiments involve techniques such as A/B testing, canary releases, or dark launches. However, neither managing multiple experiments in parallel (i.e., operating and monitoring multiple versions), nor specifying parameters for experiments (e.g., to avoid that they negatively impact each other) is a trivial task. In my research, I want to support developers and release engineers conducting experiments in an automated and data-driven way.