Regularised Zero-Variance Control Variates for High-Dimensional Variance Reduction

TL;DR: In this paper, the authors present compelling empirical evidence that the use of penalised regression techniques in the selection of high-dimensional control variates provides performance gains over the classical least squares method.

Abstract: Zero-variance control variates (ZV-CV) are a post-processing method to reduce the variance of Monte Carlo estimators of expectations using the derivatives of the log target. Once the derivatives are available, the only additional computational effort lies in solving a linear regression problem. Significant variance reductions have been achieved with this method in low dimensional examples, but the number of covariates in the regression rapidly increases with the dimension of the target. In this paper, we present compelling empirical evidence that the use of penalised regression techniques in the selection of high-dimensional control variates provides performance gains over the classical least squares method. Another type of regularisation based on using subsets of derivatives, or a priori regularisation as we refer to it in this paper, is also proposed to reduce computational and storage requirements. Several examples showing the utility and limitations of regularised ZV-CV for Bayesian inference are given. The methods proposed in this paper are accessible through the R package ZVCV.