Junction tree algorithm

Abstract: Judging by the increasing impact of machine learning on large-scale data analysis in the last decade, one can anticipate a substantial growth in diversity of the machine learning applications for "big data" over the next decade. This exciting new opportunity, however, also raises many challenges. One of them is scaling inference within and training of graphical models. Typical ways to address this scaling issue are inference by approximate message passing, stochastic gradients, and MapReduce, among others. Often, we encounter inference and training problems with symmetries and redundancies in the graph structure. A prominent example are relational models that capture complexity. Exploiting these symmetries, however, has not been considered for scaling yet. In this paper, we show that inference and training can indeed benefit from exploiting symmetries. Specifically, we show that (loopy) belief propagation (BP) can be lifted. That is, a model is compressed by grouping nodes together that send and receive identical messages so that a modified BP running on the lifted graph yields the same marginals as BP on the original one, but often in a fraction of time. By establishing a link between lifting and radix sort, we show that lifting is MapReduce-able. Still, in many if not most situations training relational models will not benefit from this (scalable) lifting: symmetries within models easily break since variables become correlated by virtue of depending asymmetrically on evidence. An appealing idea for such situations is to train and recombine local models. This breaks long-range dependencies and allows to exploit lifting within and across the local training tasks. Moreover, it naturally paves the way for the first scalable lifted training approaches based on stochastic gradients, both in an online and a MapReduced fashion. On several datasets, the online training, for instance, converges to the same quality solution over an order of magnitude faster, simply because it starts optimizing long before having seen the entire mega-example even once.

Abstract: Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating dis­ tribution that factorises into cluster poten­ tials. This includes undirected graphs, di­ rected acyclic graphs and junction trees. We derive generalised mean field equations to op­ timise the cluster potentials. We show that the method bridges the gap between the stan­ dard mean field approximation and the exact junction tree algorithm. In addition, we ad­ dress the problem of how to choose the struc­ ture and the free parameters of the approx­ imating distribution. From the generalised mean field equations we derive rules to sim­ plify the approximation in advance without affecting the potential accuracy of the model class. We also show how the method fits into some other variational approximations that are currently popular.

Abstract: This paper presents a method of predicting future human driving behavior under the condition that its resultant behavior and past observations are given. The proposed method makes use of a dynamic Bayesian network and the junction tree algorithm for probabilistic inference. The method is applied to behavior prediction for a vehicle assumed to stop at an intersection. Such a predictive system would facilitate warning and assistance to prevent dangerous activities, such as red-light violations, by allowing detection of a deviation from normal behavior.