Open AccessPosted Content
A Lambda-Calculus Foundation for Universal Probabilistic Programming
TL;DR: This work adapts the classic operational semantics of λ-calculus to a continuous setting via creating a measure space on terms and defining step-indexed approximations, and proves equivalence of big-step and small-step formulations of this distribution-based semantics.
read more
Abstract: We develop the operational semantics of an untyped probabilistic lambda-calculus with continuous distributions, as a foundation for universal probabilistic programming languages such as Church, Anglican, and Venture. Our first contribution is to adapt the classic operational semantics of lambda-calculus to a continuous setting via creating a measure space on terms and defining step-indexed approximations. We prove equivalence of big-step and small-step formulations of this distribution-based semantics. To move closer to inference techniques, we also define the sampling-based semantics of a term as a function from a trace of random samples to a value. We show that the distribution induced by integrating over all traces equals the distribution-based semantics. Our second contribution is to formalize the implementation technique of trace Markov chain Monte Carlo (MCMC) for our calculus and to show its correctness. A key step is defining sufficient conditions for the distribution induced by trace MCMC to converge to the distribution-based semantics. To the best of our knowledge, this is the first rigorous correctness proof for trace MCMC for a higher-order functional language.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Design and Implementation of Probabilistic Programming Language Anglican
David Tolpin,Jan-Willem van de Meent,Hongseok Yang,Frank Wood +3 more
- 31 Aug 2016
TL;DR: It is shown that a probabilistic functional language can be implemented efficiently and integrated tightly with a conventional functional language with only moderate computational overhead and how advanced probabilism modelling concepts are mapped naturally to the functional foundation.
138
A Convenient Category for Higher-Order Probability Theory
TL;DR: In this paper, the authors introduce the notion of quasi-Borel spaces for higher-order functions and probability, and demonstrate the use of these spaces for probabilistic programming languages.
123
Denotational validation of higher-order Bayesian inference
Adam Ścibior,Ohad Kammar,Matthijs Vákár,Sam Staton,Hongseok Yang,Yufei Cai,Klaus Ostermann,Sean K. Moss,Chris Heunen,Zoubin Ghahramani +9 more
- 27 Dec 2017
TL;DR: In this paper, a modular semantic account of Bayesian inference algorithms for probabilistic programming languages, as used in data science and machine learning, is presented, and a collection of building blocks for composing representations are developed.
84
A domain theory for statistical probabilistic programming
Matthijs Vákár,Ohad Kammar,Sam Staton +2 more
- 02 Jan 2019
TL;DR: Quasi-Borel predomains form both a model of Fiore's axiomatic domain theory and a models of Kock's synthetic measure theory, which gives an adequate denotational semantics for languages with recursive higher-order types, continuous probability distributions, and soft constraints.
Measurable Cones and Stable, Measurable Functions
TL;DR: A notion of stable and measurable map between cones endowed with measurability tests is defined and it is shown that it forms a cpo-enriched cartesian closed category that gives a denotational model of an extension of PCF supporting the main primitives of probabilistic functional programming.
54
References
Equation of state calculations by fast computing machines
TL;DR: In this article, a modified Monte Carlo integration over configuration space is used to investigate the properties of a two-dimensional rigid-sphere system with a set of interacting individual molecules, and the results are compared to free volume equations of state and a four-term virial coefficient expansion.
•Book
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference
Judea Pearl
- 01 Jan 1988
TL;DR: Probabilistic Reasoning in Intelligent Systems as mentioned in this paper is a complete and accessible account of the theoretical foundations and computational methods that underlie plausible reasoning under uncertainty, and provides a coherent explication of probability as a language for reasoning with partial belief.
17.6K
Monte Carlo Sampling Methods Using Markov Chains and Their Applications
TL;DR: A generalization of the sampling method introduced by Metropolis et al. as mentioned in this paper is presented along with an exposition of the relevant theory, techniques of application and methods and difficulties of assessing the error in Monte Carlo estimates.
17.1K
•Book
Foundations of Statistical Natural Language Processing
Christopher D. Manning,Hinrich Schütze +1 more
- 28 May 1999
TL;DR: This foundational text is the first comprehensive introduction to statistical natural language processing (NLP) to appear and provides broad but rigorous coverage of mathematical and linguistic foundations, as well as detailed discussion of statistical methods, allowing students and researchers to construct their own implementations.
Markov Chains for Exploring Posterior Distributions
TL;DR: Several Markov chain methods are available for sampling from a posterior distribution as discussed by the authors, including Gibbs sampler and Metropolis algorithm, and several strategies for constructing hybrid algorithms, which can be used to guide the construction of more efficient algorithms.
Related Papers (5)
Raven Beutner,Luke Ong +1 more
- 19 Jun 2021
Andreas Morgenstern,Manuel Gesell,Klaus Schneider +2 more
- 11 Mar 2012