Open AccessProceedings Article
Generating Efficient MCMC Kernels from Probabilistic Programs
Lingfeng Yang,Pat Hanrahan,Noah D. Goodman +2 more
- 02 Apr 2014
- pp 1068-1076
TL;DR: This work presents a technique that recovers hand-coded levels of performance from a universal probabilistic language, for the Metropolis-Hastings (MH) MCMC inference algorithm, that takes a Church program as input and traces its execution to remove computation overhead.
read more
Abstract: Universal probabilistic programming languages (such as Church [6]) trade performance for abstraction: any model can be represented compactly as an arbitrary stochastic computation, but costly online analyses are required for inference. We present a technique that recovers hand-coded levels of performance from a universal probabilistic language, for the Metropolis-Hastings (MH) MCMC inference algorithm. It takes a Church program as input and traces its execution to remove computation overhead. It then analyzes the trace for each proposal, using slicing, to identify the minimal computation needed to evaluate the MH acceptance probability. Generated incremental code is much faster than a baseline implementation (up to 600x) and usually as fast as handcoded MH kernels.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
An Introduction to Probabilistic Programming
TL;DR: This document starts with a discussion of model-based reasoning and explains why conditioning as a foundational computation is central to the fields of probabilistic machine learning and artificial intelligence, and introduces a simple first-order Probabilistic programming language whose programs define static-computation-graph, finite-variable-cardinality models.
204
Design and Implementation of Probabilistic Programming Language Anglican
David Tolpin,Jan-Willem van de Meent,Hongseok Yang,Frank Wood +3 more
- 31 Aug 2016
TL;DR: It is shown that a probabilistic functional language can be implemented efficiently and integrated tightly with a conventional functional language with only moderate computational overhead and how advanced probabilism modelling concepts are mapped naturally to the functional foundation.
138
•Dissertation
Automating inference, learning, and design using probabilistic programming
Tom Rainforth
- 01 Jan 2017
TL;DR: The aim of this paper is to propose a novel approach toference called Automated Variational Inference for Probabilistic Programming, which allows programmers to specify a stochastic process using syntax that resembles modern programming lan 2.
42
•Posted Content
Swift: Compiled Inference for Probabilistic Programming Languages
TL;DR: Swift is described, a compiler for the BLOG PPL that incorporates optimizations that eliminate interpretation overhead, maintain dynamic dependencies efficiently, and handle memory management for possible worlds of varying sizes.
27
•Proceedings Article
C3: Lightweight Incrementalized MCMC for Probabilistic Programs using Continuations and Callsite Caching
Daniel Ritchie,Andreas Stuhlmüller,Noah D. Goodman +2 more
- 02 May 2016
TL;DR: C3 is a new extension to the lightweight approach to implementing MCMC for probabilistic programming languages, which enables efficient, incrementalized re-execution of MH proposals and reduces proposal runtime by 20-100x, in some cases reducing runtime complexity from linear in model size to constant.
References
•Book
Compilers: Principles, Techniques, and Tools
Alfred V. Aho,Ravi Sethi,Jeffrey D. Ullman +2 more
- 01 Jan 1986
TL;DR: This book discusses the design of a Code Generator, the role of the Lexical Analyzer, and other topics related to code generation and optimization.
9.7K
LLVM: a compilation framework for lifelong program analysis & transformation
Chris Lattner,Vikram Adve +1 more
- 20 Mar 2004
TL;DR: The design of the LLVM representation and compiler framework is evaluated in three ways: the size and effectiveness of the representation, including the type information it provides; compiler performance for several interprocedural problems; and illustrative examples of the benefits LLVM provides for several challenging compiler problems.
Hierarchical Dirichlet Processes
TL;DR: This work considers problems involving groups of data where each observation within a group is a draw from a mixture model and where it is desirable to share mixture components between groups, and considers a hierarchical model, specifically one in which the base measure for the childDirichlet processes is itself distributed according to a Dirichlet process.
Program Slicing
TL;DR: Program slicing as mentioned in this paper is a method for automatically decomposing programs by analyzing their data flow and control flow. But it is not a technique for finding statement-minimal slices, as it is in general unsolvable, but using data flow analysis is sufficient to find approximate slices.
3.3K
Program slicing
Keith Gallagher,David Binkley +1 more
- 01 Sep 2008
TL;DR: Applications of program slicing are surveyed, ranging from its first use as a debugging technique to current applications in property verification using finite state models, and a summary of research challenges for the slicing community is discussed.
2.8K
Related Papers (5)
Andrew D. Gordon,Thomas A. Henzinger,Aditya V. Nori,Sriram K. Rajamani +3 more
- 31 May 2014
Avi Pfeffer
- 04 Aug 2001