Open AccessProceedings Article
Non-conjugate Variational Message Passing for Multinomial and Binary Regression
David A. Knowles,Tom Minka +1 more
- 12 Dec 2011
- Vol. 24, pp 1701-1709
TL;DR: An extension to VMP which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are calculated, and integrating into an existing message-passing framework: Infer.NET is proposed.
read more
Abstract: Variational Message Passing (VMP) is an algorithmic implementation of the Variational Bayes (VB) method which applies only in the special case of conjugate exponential family models. We propose an extension to VMP, which we refer to as Non-conjugate Variational Message Passing (NCVMP) which aims to alleviate this restriction while maintaining modularity, allowing choice in how expectations are calculated, and integrating into an existing message-passing framework: Infer.NET. We demonstrate NCVMP on logistic binary and multinomial regression. In the multinomial case we introduce a novel variational bound for the soft-max factor which is tighter than other commonly used bounds whilst maintaining computational tractability.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Variational Inference: A Review for Statisticians
TL;DR: For instance, mean-field variational inference as discussed by the authors approximates probability densities through optimization, which is used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling.
Variational Inference: A Review for Statisticians
TL;DR: Variational inference (VI), a method from machine learning that approximates probability densities through optimization, is reviewed and a variant that uses stochastic optimization to scale up to massive data is derived.
2.9K
Stochastic variational inference
TL;DR: Stochastic variational inference lets us apply complex Bayesian models to massive data sets, and it is shown that the Bayesian nonparametric topic model outperforms its parametric counterpart.
•Proceedings Article
Black Box Variational Inference
Rajesh Ranganath,Sean Gerrish,David M. Blei +2 more
- 02 Apr 2014
TL;DR: This paper presents a "black box" variational inference algorithm, one that can be quickly applied to many models with little additional derivation, based on a stochastic optimization of the variational objective where the noisy gradient is computed from Monte Carlo samples from the Variational distribution.
A model of text for experimentation in the social sciences
TL;DR: A hierarchical mixed membership model for analyzing topical content of documents, in which mixing weights are parameterized by observed covariates is posit, enabling researchers to introduce elements of the experimental design that informed document collection into the model, within a generally applicable framework.
652
References
•Proceedings Article
Expectation propagation for approximate Bayesian inference
Tom Minka
- 02 Aug 2001
TL;DR: Expectation Propagation approximates the belief states by only retaining expectations, such as mean and varitmce, and iterates until these expectations are consistent throughout the network, which makes it applicable to hybrid networks with discrete and continuous nodes.
1.9K
•Posted Content
Expectation Propagation for approximate Bayesian inference
TL;DR: Expectation Propagation (EP) as mentioned in this paper is a deterministic approximation technique in Bayesian networks that unifies two previous techniques: assumed-density filtering, an extension of the Kalman filter, and loopy belief propagation.
1.3K
A correlated topic model of Science
David M. Blei,John Lafferty +1 more
TL;DR: The correlated topic model (CTM) is developed, where the topic proportions exhibit correlation via the logistic normal distribution, and it is demonstrated its use as an exploratory tool of large document collections.
1.3K
A correlated topic model of Science
David M. Blei,John Lafferty +1 more
TL;DR: The correlated topic model (CTM) as mentioned in this paper uses the logistic normal distribution to model the topic proportions, which is a variant of the Dirichlet distribution used in LDA.
•Proceedings Article
A Variational Baysian Framework for Graphical Models
Hagai Attias
- 29 Nov 1999
TL;DR: This paper presents a novel practical framework for Bayesian model averaging and model selection in probabilistic graphical models that approximates full posterior distributions over model parameters and structures, as well as latent variables, in an analytical manner.