Using Sample-based Representations Under Communications Constraints

TL;DR: A novel density approximation method based on KD-tree multiscale representations which enables the communications cost and a bound on error to be balanced eciently is proposed and several empirical examples demonstrating the method’s utility in collaborative, distributed signal processing under bandwidth or power constraints are shown.

Abstract: In many applications, particularly power-constrained sensor networks, it is important to conserve the amount of data exchanged while maximizing the utility of that data for some inference task. Broadly, this tradeo has two major cost components—the representation’s size (in distributed networks, the communications cost) and the error incurred by its use (the inference cost). We analyze this tradeo for a particular problem: communicating a particle-based representation (and more generally, a Gaussian mixture or kernel density estimate). We begin by characterizing the exact communication cost of these representations, noting that it is less than might be suggested by traditional communications theory due to the invariance of the representation to reordering. We describe the optimal, lossless encoder when the generating distribution is known, and pose a sub-optimal encoder which still benefits from reordering invariance. However, lossless encoding may not be sucient. We describe one reasonable measure of error for distribution-based messages and its consequences for inference in an acyclic network, and propose a novel density approximation method based on KD-tree multiscale representations which enables the communications cost and a bound on error to be balanced eciently. We show several empirical examples demonstrating the method’s utility in collaborative, distributed signal processing under bandwidth or power constraints.