Scalable Hamiltonian Monte Carlo via Surrogate Methods

Abstract

Markov chain Monte Carlo methods have been widely used in Bayesian inference involving intractable probabilistic models. However, simple MCMC algorithms are notorious for their lack of computational efficiency in complex, high-dimensional models and poor scaling to large data sets. In recent years, many advanced MCMC methods have been proposed that utilize geometrical and statistical quantities from the model in order to explore the target distribution more effectively. The gain in the efficacy of exploration, however, often comes at a significant computational cost which hinders their application to problems with large data sets or complex likelihoods. In practice, it remains challenging to design scalable MCMC algorithms that can balance computational complexity and exploration efficacy well. To address this issue, some recent algorithms rely on stochastic gradient methods by approximating full data gradients using mini-batches of data. In contrast, this thesis focuses on accelerating the computation of MCMC samplers based on various surrogate methods via exploring the regularity of the target distribution.We start with a precomputing strategy that can be used to build efficient surrogates in relatively low-dimension parameter spaces. We then propose a random network surrogate architecture which can effectively capture the collective properties of large data sets or complex models with scalability, flexibility and efficiency. Finally, we provide a variational perspective for our random network surrogate methods and propose an approximate inference framework that combines the advantages of both variational Bayes and Markov chain Monte Carlo methods. The properties and efficiency of our proposed methods are demonstrated on a variety of synthetic and real-world data problems.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 90,593

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

  • Only published works are available at libraries.

Similar books and articles

Inductive inference in the limit of empirically adequate theories.Bernhard Lauth - 1995 - Journal of Philosophical Logic 24 (5):525 - 548.
Why Monte Carlo Simulations Are Inferences and Not Experiments.Claus Beisbart & John D. Norton - 2012 - International Studies in the Philosophy of Science 26 (4):403-422.

Analytics

Added to PP
2017-04-10

Downloads
0

6 months
0

Historical graph of downloads

Sorry, there are not enough data points to plot this chart.
How can I increase my downloads?

Citations of this work

No citations found.

Add more citations

References found in this work

No references found.

Add more references