Download or read book Methods for Scalable Markov Chain Monte Carlo Inference written by Dangna Li and published by . This book was released on 2018 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation contains three methods aiming at improving the scalability of Markov Chain Monte Carlo (MCMC) sampling. The first method addresses the computation bottleneck of Metropolis-Hastings in Bayesian models with a large number of observations. As an extension to the first method, we have developed a new MCMC algorithm to handle multimodal high-dimensional posterior distributions with a large number of observations. Our objective with the third method is to improve the efficiency of MH in high dimension, by designing a novel local MH move to make an effective use of the gradient information of the log target density. In the first part, we propose a general framework of performing MCMC with only a mini-batch of data. We show that by estimating the MH ratio with only a subset of data, one can sample from the true posterior raised to a known temperature. We show by experiments that our algorithm, ``Mini-batch Tempered MCMC'', can efficiently explore the landscape of a multimodal posterior distribution. In addition, based on the Equi-Energy sampler \citep{kou2006discussion}, we propose a new MCMC algorithm, which enables exact sampling from high-dimensional multimodal posteriors with well-separated modes. In the second part, we introduce a novel local MH move which we shall refer to as the ``Cone Move''. Our goal is to address a crucial limitation of Langevin Dynamics, namely its inherent inefficiency in utilizing the gradient information in exploring the target distribution. Compared with Langevin Dynamics, Cone Move makes better use of the gradient information and thus generates more targeted moves along the shape of the target distribution. We propose a new MCMC algorithm, the Cone Sampler, which adapts to the geometry of the target density by alternating between Cone Move and Langevin Dynamics. We demonstrate the effectiveness of Cone Sampler by testing its performance on challenging sampling problems in high dimensions.
Download or read book Scalable Hamiltonian Monte Carlo Via Surrogate Methods written by Cheng Zhang and published by . This book was released on 2017 with total page 154 pages. Available in PDF, EPUB and Kindle. Book excerpt: Markov chain Monte Carlo (MCMC) methods have been widely used in Bayesian inference involving intractable probabilistic models. However, simple MCMC algorithms (e.g., random walk Metropolis and Gibbs sampling) are notorious for their lack of computational efficiency in complex, high-dimensional models and poor scaling to large data sets. In recent years, many advanced MCMC methods (e.g., Hamiltonian Monte Carlo and Riemannian Manifold Hamiltonian Monte Carlo) have been proposed that utilize geometrical and statistical quantities from the model in order to explore the target distribution more effectively. The gain in the efficacy of exploration, however, often comes at a significant computational cost which hinders their application to problems with large data sets or complex likelihoods.In practice, it remains challenging to design scalable MCMC algorithms that can balance computational complexity and exploration efficacy well. To address this issue, some recent algorithms rely on stochastic gradient methods by approximating full data gradients using mini-batches of data. In contrast, this thesis focuses on accelerating the computation of MCMC samplers based on various surrogate methods via exploring the regularity of the target distribution.We start with a precomputing strategy that can be used to build efficient surrogates in relatively low-dimension parameter spaces. We then propose a random network surrogate architecture which can effectively capture the collective properties of large data sets or complex models with scalability, flexibility and efficiency. Finally, we provide a variational perspective for our random network surrogate methods and propose an approximate inference framework that combines the advantages of both variational Bayes and Markov chain Monte Carlo methods. The properties and efficiency of our proposed methods are demonstrated on a variety of synthetic and real-world data problems.
Download or read book Markov Chain Monte Carlo written by Dani Gamerman and published by CRC Press. This book was released on 2006-05-10 with total page 352 pages. Available in PDF, EPUB and Kindle. Book excerpt: While there have been few theoretical contributions on the Markov Chain Monte Carlo (MCMC) methods in the past decade, current understanding and application of MCMC to the solution of inference problems has increased by leaps and bounds. Incorporating changes in theory and highlighting new applications, Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference, Second Edition presents a concise, accessible, and comprehensive introduction to the methods of this valuable simulation technique. The second edition includes access to an internet site that provides the code, written in R and WinBUGS, used in many of the previously existing and new examples and exercises. More importantly, the self-explanatory nature of the codes will enable modification of the inputs to the codes and variation on many directions will be available for further exploration. Major changes from the previous edition: · More examples with discussion of computational details in chapters on Gibbs sampling and Metropolis-Hastings algorithms · Recent developments in MCMC, including reversible jump, slice sampling, bridge sampling, path sampling, multiple-try, and delayed rejection · Discussion of computation using both R and WinBUGS · Additional exercises and selected solutions within the text, with all data sets and software available for download from the Web · Sections on spatial models and model adequacy The self-contained text units make MCMC accessible to scientists in other disciplines as well as statisticians. The book will appeal to everyone working with MCMC techniques, especially research and graduate statisticians and biostatisticians, and scientists handling data and formulating models. The book has been substantially reinforced as a first reading of material on MCMC and, consequently, as a textbook for modern Bayesian computation and Bayesian inference courses.
Download or read book Advanced Markov Chain Monte Carlo Methods written by Faming Liang and published by John Wiley & Sons. This book was released on 2011-07-05 with total page 308 pages. Available in PDF, EPUB and Kindle. Book excerpt: Markov Chain Monte Carlo (MCMC) methods are now an indispensable tool in scientific computing. This book discusses recent developments of MCMC methods with an emphasis on those making use of past sample information during simulations. The application examples are drawn from diverse fields such as bioinformatics, machine learning, social science, combinatorial optimization, and computational physics. Key Features: Expanded coverage of the stochastic approximation Monte Carlo and dynamic weighting algorithms that are essentially immune to local trap problems. A detailed discussion of the Monte Carlo Metropolis-Hastings algorithm that can be used for sampling from distributions with intractable normalizing constants. Up-to-date accounts of recent developments of the Gibbs sampler. Comprehensive overviews of the population-based MCMC algorithms and the MCMC algorithms with adaptive proposals. This book can be used as a textbook or a reference book for a one-semester graduate course in statistics, computational biology, engineering, and computer sciences. Applied or theoretical researchers will also find this book beneficial.
Download or read book Handbook of Markov Chain Monte Carlo written by Steve Brooks and published by CRC Press. This book was released on 2011-05-10 with total page 620 pages. Available in PDF, EPUB and Kindle. Book excerpt: Since their popularization in the 1990s, Markov chain Monte Carlo (MCMC) methods have revolutionized statistical computing and have had an especially profound impact on the practice of Bayesian statistics. Furthermore, MCMC methods have enabled the development and use of intricate models in an astonishing array of disciplines as diverse as fisherie
Download or read book Markov Chain Monte Carlo in Practice written by W.R. Gilks and published by CRC Press. This book was released on 1995-12-01 with total page 538 pages. Available in PDF, EPUB and Kindle. Book excerpt: In a family study of breast cancer, epidemiologists in Southern California increase the power for detecting a gene-environment interaction. In Gambia, a study helps a vaccination program reduce the incidence of Hepatitis B carriage. Archaeologists in Austria place a Bronze Age site in its true temporal location on the calendar scale. And in France, researchers map a rare disease with relatively little variation. Each of these studies applied Markov chain Monte Carlo methods to produce more accurate and inclusive results. General state-space Markov chain theory has seen several developments that have made it both more accessible and more powerful to the general statistician. Markov Chain Monte Carlo in Practice introduces MCMC methods and their applications, providing some theoretical background as well. The authors are researchers who have made key contributions in the recent development of MCMC methodology and its application. Considering the broad audience, the editors emphasize practice rather than theory, keeping the technical content to a minimum. The examples range from the simplest application, Gibbs sampling, to more complex applications. The first chapter contains enough information to allow the reader to start applying MCMC in a basic way. The following chapters cover main issues, important concepts and results, techniques for implementing MCMC, improving its performance, assessing model adequacy, choosing between models, and applications and their domains. Markov Chain Monte Carlo in Practice is a thorough, clear introduction to the methodology and applications of this simple idea with enormous potential. It shows the importance of MCMC in real applications, such as archaeology, astronomy, biostatistics, genetics, epidemiology, and image analysis, and provides an excellent base for MCMC to be applied to other fields as well.
Download or read book A Scalable Multi chain Markov Chain Monte Carlo Method for Inverting Subsurface Hydraulic and Geological Properties written by and published by . This book was released on 2015 with total page 14 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Scaling MCMC Inference and Belief Propagation to Large Dense Graphical Models written by Sameer Singh and published by . This book was released on 2014 with total page 185 pages. Available in PDF, EPUB and Kindle. Book excerpt: With the physical constraints of semiconductor-based electronics becoming increasingly limiting in the past decade, single-core CPUs have given way to multi-core and distributed computing platforms. At the same time, access to large data collections is progressively becoming commonplace due to the lowering cost of storage and bandwidth. Traditional machine learning paradigms that have been designed to operate sequentially on single processor architectures seem destined to become obsolete in this world of multi-core, multi-node systems and massive data sets. Inference for graphical models is one such example for which most existing algorithms are sequential in nature and are difficult to scale using parallel computations. Further, modeling large datasets leads to an escalation in the number of variables, factors, domains, and the density of the models, all of which have a substantial impact on the computational and storage complexity of inference. To achieve scalability, existing techniques impose strict independence assumptions on the model, resulting in tractable inference at the expense of expressiveness, and therefore of accuracy and utility, of the model. Motivated by the need to scale inference to large, dense graphical models, in this thesis we explore approximations to Markov chain Monte Carlo (MCMC) and belief propagation (BP) that induce dynamic sparsity in the model to utilize parallelism. In particular, since computations over some factors, variables, and values are more important than over others at different stages of inference, proposed approximations that prioritize and parallelize such computations facilitate efficient inference. First, we show that a synchronously distributed MCMC algorithm that uses dynamic partitioning of the model achieves scalable inference. We then identify bottlenecks in the synchronous architecture, and demonstrate that a collection of MCMC techniques that use asynchronous updates are able to address these drawbacks. For large domains and high-order factors, we find that dynamically inducing sparsity in variable domains, results in scalable belief propagation that enables joint inference. We also show that formulating distributed BP and joint inference as generalized BP on cluster graphs, and by using cluster message approximations, provides significantly lower communication cost and running time. With these tools for inference in hand, we are able to tackle entity tagging, relation extraction, entity resolution, cross-document coreference, joint inference, and other information extraction tasks over large text corpora.
Download or read book Acceleration Strategies of Markov Chain Monte Carlo for Bayesian Computation written by Chang-Ye Wu and published by . This book was released on 2018 with total page 110 pages. Available in PDF, EPUB and Kindle. Book excerpt: MCMC algorithms are difficult to scale, since they need to sweep over the whole data set at each iteration, which prohibits their applications in big data settings. Roughly speaking, all scalable MCMC algorithms can be divided into two categories: divide-and-conquer methods and subsampling methods. The aim of this project is to reduce the computing time induced by complex or largelikelihood functions.
Download or read book Scalable and Efficient Probabilistic Topic Model Inference for Textual Data written by Måns Magnusson and published by Linköping University Electronic Press. This book was released on 2018-04-27 with total page 75 pages. Available in PDF, EPUB and Kindle. Book excerpt: Probabilistic topic models have proven to be an extremely versatile class of mixed-membership models for discovering the thematic structure of text collections. There are many possible applications, covering a broad range of areas of study: technology, natural science, social science and the humanities. In this thesis, a new efficient parallel Markov Chain Monte Carlo inference algorithm is proposed for Bayesian inference in large topic models. The proposed methods scale well with the corpus size and can be used for other probabilistic topic models and other natural language processing applications. The proposed methods are fast, efficient, scalable, and will converge to the true posterior distribution. In addition, in this thesis a supervised topic model for high-dimensional text classification is also proposed, with emphasis on interpretable document prediction using the horseshoe shrinkage prior in supervised topic models. Finally, we develop a model and inference algorithm that can model agenda and framing of political speeches over time with a priori defined topics. We apply the approach to analyze the evolution of immigration discourse in the Swedish parliament by combining theory from political science and communication science with a probabilistic topic model. Probabilistiska ämnesmodeller (topic models) är en mångsidig klass av modeller för att estimera ämnessammansättningar i större corpusar. Applikationer finns i ett flertal vetenskapsområden som teknik, naturvetenskap, samhällsvetenskap och humaniora. I denna avhandling föreslås nya effektiva och parallella Markov Chain Monte Carlo algoritmer för Bayesianska ämnesmodeller. De föreslagna metoderna skalar väl med storleken på corpuset och kan användas för flera olika ämnesmodeller och liknande modeller inom språkteknologi. De föreslagna metoderna är snabba, effektiva, skalbara och konvergerar till den sanna posteriorfördelningen. Dessutom föreslås en ämnesmodell för högdimensionell textklassificering, med tonvikt på tolkningsbar dokumentklassificering genom att använda en kraftigt regulariserande priorifördelningar. Slutligen utvecklas en ämnesmodell för att analyzera "agenda" och "framing" för ett förutbestämt ämne. Med denna metod analyserar vi invandringsdiskursen i Sveriges Riksdag över tid, genom att kombinera teori från statsvetenskap, kommunikationsvetenskap och probabilistiska ämnesmodeller.
Download or read book Markov Chain Monte Carlo Simulations and Their Statistical Analysis written by Bernd A Berg and published by World Scientific Publishing Company. This book was released on 2004-10-01 with total page 380 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book teaches modern Markov chain Monte Carlo (MC) simulation techniques step by step. The material should be accessible to advanced undergraduate students and is suitable for a course. It ranges from elementary statistics concepts (the theory behind MC simulations), through conventional Metropolis and heat bath algorithms, autocorrelations and the analysis of the performance of MC algorithms, to advanced topics including the multicanonical approach, cluster algorithms and parallel computing. Therefore, it is also of interest to researchers in the field. The book relates the theory directly to Web-based computer code. This allows readers to get quickly started with their own simulations and to verify many numerical examples easily. The present code is in Fortran 77, for which compilers are freely available. The principles taught are important for users of other programming languages, like C or C++.
Download or read book Current Trends in Bayesian Methodology with Applications written by Satyanshu K. Upadhyay and published by CRC Press. This book was released on 2015-05-21 with total page 674 pages. Available in PDF, EPUB and Kindle. Book excerpt: Collecting Bayesian material scattered throughout the literature, Current Trends in Bayesian Methodology with Applications examines the latest methodological and applied aspects of Bayesian statistics. The book covers biostatistics, econometrics, reliability and risk analysis, spatial statistics, image analysis, shape analysis, Bayesian computation, clustering, uncertainty assessment, high-energy astrophysics, neural networking, fuzzy information, objective Bayesian methodologies, empirical Bayes methods, small area estimation, and many more topics. Each chapter is self-contained and focuses on a Bayesian methodology. It gives an overview of the area, presents theoretical insights, and emphasizes applications through motivating examples. This book reflects the diversity of Bayesian analysis, from novel Bayesian methodology, such as nonignorable response and factor analysis, to state-of-the-art applications in economics, astrophysics, biomedicine, oceanography, and other areas. It guides readers in using Bayesian techniques for a range of statistical analyses.
Download or read book Scalable MCMC for Large Data Problems Using Data Subsampling and the Difference Estimator written by Matias Quiroz and published by . This book was released on 2016 with total page 32 pages. Available in PDF, EPUB and Kindle. Book excerpt: We propose a generic Markov Chain Monte Carlo (MCMC) algorithm to speed up computations for datasets with many observations. A key feature of our approach is the use of the highly effcient difference estimator from the survey sampling literature to estimate the log-likelihood accurately using only a small fraction of the data. Our algorithm improves on the O(n) complexity of regular MCMC by operating over local data clusters instead of the full sample when computing the likelihood. The likelihood estimate is used in a Pseudo-marginal framework to sample from a perturbed posterior which is within O(m -1/2) of the true posterior, where m is the subsample size. The method is applied to a logistic regression model to predict firm bankruptcy for a large data set. We document a significant speed up in comparison to the standard MCMC on the full dataset.
Download or read book Scalable Geometric Markov Chain Monte Carlo written by Yichuan Zhang and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Monte Carlo Methods written by Neal Noah Madras and published by American Mathematical Soc.. This book was released on 2000 with total page 238 pages. Available in PDF, EPUB and Kindle. Book excerpt: This volume contains the proceedings of the Workshop on Monte Carlo Methods held at The Fields Institute for Research in Mathematical Sciences (Toronto, 1998). The workshop brought together researchers in physics, statistics, and probability. The papers in this volume - of the invited speakers and contributors to the poster session - represent the interdisciplinary emphasis of the conference. Monte Carlo methods have been used intensively in many branches of scientific inquiry. Markov chain methods have been at the forefront of much of this work, serving as the basis of many numerical studies in statistical physics and related areas since the Metropolis algorithm was introduced in 1953. Statisticians and theoretical computer scientists have used these methods in recent years, working on different fundamental research questions, yet using similar Monte Carlo methodology. This volume focuses on Monte Carlo methods that appear to have wide applicability and emphasizes new methods, practical applications and theoretical analysis. It will be of interest to researchers and graduate students who study and/or use Monte Carlo methods in areas of probability, statistics, theoretical physics, or computer science.
Download or read book Markov Chain Monte Carlo written by W. S. Kendall and published by World Scientific. This book was released on 2005 with total page 239 pages. Available in PDF, EPUB and Kindle. Book excerpt: Markov Chain Monte Carlo (MCMC) originated in statistical physics, but has spilled over into various application areas, leading to a corresponding variety of techniques and methods. That variety stimulates new ideas and developments from many different places, and there is much to be gained from cross-fertilization. This book presents five expository essays by leaders in the field, drawing from perspectives in physics, statistics and genetics, and showing how different aspects of MCMC come to the fore in different contexts. The essays derive from tutorial lectures at an interdisciplinary program at the Institute for Mathematical Sciences, Singapore, which exploited the exciting ways in which MCMC spreads across different disciplines.
Download or read book Dynamic Shrinkage Priors for Large Time varying Parameter Regressions Using Scalable Markov Chain Monte Carlo Methods written by Niko Hauzenberger and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt:
