Download or read book Bayesian Decision Tree Ensembles for Nonparametric Problems written by Yinpu Li and published by . This book was released on 2021 with total page 127 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian additive regression trees(BART) provides flexible approach to fitting a variety of regression models while avoiding strong parametric assumptions. The sum-of-trees model is embedded in a Bayesian inferential framework to support uncertainty quantification and provide a principled approach to regularization through prior specification. This dissertation presents two extensions of BART. The first extension builds a Bayesian nonparametric model for conditional density regression. The generative model used is based on rejection sampling from a base model. Like other BART models, this model is flexible, has a default prior specification, and is computationally convenient. To address the distinguished role of the response in our BART extension, we further introduce an approach to targeted smoothing of BART models which is possibly of independent interest. We study the proposed model theoretically and provide sufficient conditions for the posterior distributions to concentrate at close to minimax optimal rate adaptively over smoothness classes in the high-dimensional regime in which many predictors are irrelevant. To fit our model, we propose a data augmentation algorithm which allows for existing BART samplers to be extended with minimal effort. We illustrate the performance of our methodology on simulated data and use it to study the relationship between education and body mass index using data from the medical expenditure panel survey(MEPS). The second extension is a Bayesian nonparametric estimation of a survival time subject to right-censoring in the presence of potentially high-dimensional predictors. We argue that several approaches, such as random survival forests and existing Bayesian nonparametric approaches, possess several drawbacks, including: computational difficulties; lack of known theoretical properties; and ineffectiveness at filtering out irrelevant predictors. We propose two models based on the Bayesian additive regression trees (BART) framework. The first, Modulated BART (MBART), is fully-nonparametric and models the failure time as the first occurrence of a non-homogeneous Poisson process. The second, CoxBART, uses a Bayesian implementation of Cox's partial likelihood. These models are adapted to high-dimensional predictors, have default prior specifications, and require simple modifications of existing BART methods to implement. We show the effectiveness of these methods on simulated and benchmark datasets. We also establish that, for a simplified variant of MBART, the posterior distribution contracts at a near-minimax optimal rate in a high-dimensional sparse asymptotic regime.

Download or read book Handbook of Bayesian Variable Selection written by Mahlet G. Tadesse and published by CRC Press. This book was released on 2021-12-24 with total page 762 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian variable selection has experienced substantial developments over the past 30 years with the proliferation of large data sets. Identifying relevant variables to include in a model allows simpler interpretation, avoids overfitting and multicollinearity, and can provide insights into the mechanisms underlying an observed phenomenon. Variable selection is especially important when the number of potential predictors is substantially larger than the sample size and sparsity can reasonably be assumed. The Handbook of Bayesian Variable Selection provides a comprehensive review of theoretical, methodological and computational aspects of Bayesian methods for variable selection. The topics covered include spike-and-slab priors, continuous shrinkage priors, Bayes factors, Bayesian model averaging, partitioning methods, as well as variable selection in decision trees and edge selection in graphical models. The handbook targets graduate students and established researchers who seek to understand the latest developments in the field. It also provides a valuable reference for all interested in applying existing methods and/or pursuing methodological extensions. Features: Provides a comprehensive review of methods and applications of Bayesian variable selection. Divided into four parts: Spike-and-Slab Priors; Continuous Shrinkage Priors; Extensions to various Modeling; Other Approaches to Bayesian Variable Selection. Covers theoretical and methodological aspects, as well as worked out examples with R code provided in the online supplement. Includes contributions by experts in the field. Supported by a website with code, data, and other supplementary material

Download or read book Bayesian Nonparametrics via Neural Networks written by Herbert K. H. Lee and published by SIAM. This book was released on 2004-01-01 with total page 106 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian Nonparametrics via Neural Networks is the first book to focus on neural networks in the context of nonparametric regression and classification, working within the Bayesian paradigm. Its goal is to demystify neural networks, putting them firmly in a statistical context rather than treating them as a black box. This approach is in contrast to existing books, which tend to treat neural networks as a machine learning algorithm instead of a statistical model. Once this underlying statistical model is recognized, other standard statistical techniques can be applied to improve the model. The Bayesian approach allows better accounting for uncertainty. This book covers uncertainty in model choice and methods to deal with this issue, exploring a number of ideas from statistics and machine learning. A detailed discussion on the choice of prior and new noninformative priors is included, along with a substantial literature review. Written for statisticians using statistical terminology, Bayesian Nonparametrics via Neural Networks will lead statisticians to an increased understanding of the neural network model and its applicability to real-world problems.

Download or read book Bayesian Modeling and Computation in Python written by Osvaldo A. Martin and published by CRC Press. This book was released on 2021-12-28 with total page 420 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian Modeling and Computation in Python aims to help beginner Bayesian practitioners to become intermediate modelers. It uses a hands on approach with PyMC3, Tensorflow Probability, ArviZ and other libraries focusing on the practice of applied statistics with references to the underlying mathematical theory. The book starts with a refresher of the Bayesian Inference concepts. The second chapter introduces modern methods for Exploratory Analysis of Bayesian Models. With an understanding of these two fundamentals the subsequent chapters talk through various models including linear regressions, splines, time series, Bayesian additive regression trees. The final chapters include Approximate Bayesian Computation, end to end case studies showing how to apply Bayesian modelling in different settings, and a chapter about the internals of probabilistic programming languages. Finally the last chapter serves as a reference for the rest of the book by getting closer into mathematical aspects or by extending the discussion of certain topics. This book is written by contributors of PyMC3, ArviZ, Bambi, and Tensorflow Probability among other libraries.

Download or read book Bayesian Networks and Decision Graphs written by Thomas Dyhre Nielsen and published by Springer Science & Business Media. This book was released on 2009-03-17 with total page 457 pages. Available in PDF, EPUB and Kindle. Book excerpt: This is a brand new edition of an essential work on Bayesian networks and decision graphs. It is an introduction to probabilistic graphical models including Bayesian networks and influence diagrams. The reader is guided through the two types of frameworks with examples and exercises, which also give instruction on how to build these models. Structured in two parts, the first section focuses on probabilistic graphical models, while the second part deals with decision graphs, and in addition to the frameworks described in the previous edition, it also introduces Markov decision process and partially ordered decision problems.

Download or read book Bayesian Additive Regression Trees for Multivariate Responses written by Seungha Um and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation introduces a nonparametric regression approach for multivariate skewed responses using Bayesian additive regression trees (BART). Existing BART methods use ensembles of decision trees to model a mean function, and have become popular recently due to their high prediction accuracy and ease of use. The usual assumption of a univariate Gaussian error distribution, however, is restrictive in many biomedical applications. Motivated by an oral healthy study, we provide a useful extension of BART, called the skewBART model, to address this problem. We then extend skewBART to allow for multivariate responses, with information shared across the decision trees associated with different responses within the same subject. The methodology accommodates within subject association and allows different skewness parameters for different responses. We illustrate the benefit of our multivariate skewBART model by analyzing the oral health study with highly skewed multiple responses. Secondly, BART model for analyzing mixed correlated continuous and count responses is proposed in this dissertation. Although Bayesian approaches are favored to model the uncertainty of estimation and to incorporate prior information, the limitation of developing simple and efficient algorithms for posterior computation has made Bayesian analysis of counts underdeveloped. However, recent latent variable construction based on Polya-Gamma random variables makes Bayesian inference appealing in the negative-binomial model with efficient sampling derived by exploiting conditional conjugacy. By utilizing Polya-Gamma latent variable construction, the proposed BART model accounts for associations between count and continuous responses. Moreover, an efficient Gibbs sampling is derived by exploiting conditional conjugacy since the negative binomial likelihood can be represented as a mixture of normals with Polya-Gamma mixing distribution. The utility of the models is demonstrated by analyzing the oral health study with mixed multiple responses.

Download or read book Decision Analysis written by J. Q. Smith and published by Chapman & Hall. This book was released on 1988 with total page 156 pages. Available in PDF, EPUB and Kindle. Book excerpt: Cloth edition, $47.50.

Download or read book Nonparametric Bayesian Models for Unsupervised Learning written by Pu Wang and published by . This book was released on 2011 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Unsupervised learning is an important topic in machine learning. In particular, clustering is an unsupervised learning problem that arises in a variety of applications for data analysis and mining. Unfortunately, clustering is an ill-posed problem and, as such, a challenging one: no ground-truth that can be used to validate clustering results is available. Two issues arise as a consequence. Various clustering algorithms embed their own bias resulting from different optimization criteria. As a result, each algorithm may discover different patterns in a given dataset. The second issue concerns the setting of parameters. In clustering, parameter setting controls the characterization of individual clusters, and the total number of clusters in the data. Clustering ensembles have been proposed to address the issue of different biases induced by various algorithms. Clustering ensembles combine different clustering results, and can provide solutions that are robust against spurious elements in the data. Although clustering ensembles provide a significant advance, they do not address satisfactorily the model selection and the parameter tuning problem. Bayesian approaches have been applied to clustering to address the parameter tuning and model selection issues. Bayesian methods provide a principled way to address these problems by assuming prior distributions on model parameters. Prior distributions assign low probabilities to parameter values which are unlikely. Therefore they serve as regularizers for modeling parameters, and can help avoid over-fitting. In addition, the marginal likelihood is used by Bayesian approaches as the criterion for model selection. Although Bayesian methods provide a principled way to perform parameter tuning and model selection, the key question \How many clusters?" is still open. This is a fundamental question for model selection. A special kind of Bayesian methods, nonparametric Bayesian approaches, have been proposed to address this important model selection issue. Unlike parametric Bayesian models, for which the number of parameters is finite and fixed, nonparametric Bayesian models allow the number of parameters to grow with the number of observations. After observing the data, nonparametric Bayesian models t the data with finite dimensional parameters. An additional issue with clustering is high dimensionality. High-dimensional data pose a difficult challenge to the clustering process. A common scenario with high-dimensional data is that clusters may exist in different subspaces comprised of different combinations of features (dimensions). In other words, data points in a cluster may be similar to each other along a subset of dimensions, but not in all dimensions. People have proposed subspace clustering techniques, a.k.a. co-clustering or bi-clustering, to address the dimensionality issue (here, I use the term co-clustering). Like clustering, also co-clustering suffers from the ill-posed nature and the lack of ground-truth to validate the results. Although attempts have been made in the literature to address individually the major issues related to clustering, no previous work has addressed them jointly. In my dissertation I propose a unified framework that addresses all three issues at the same time. I designed a nonparametric Bayesian clustering ensemble (NBCE) approach, which assumes that multiple observed clustering results are generated from an unknown consensus clustering. The under- lying distribution is assumed to be a mixture distribution with a nonparametric Bayesian prior, i.e., a Dirichlet Process. The number of mixture components, a.k.a. the number of consensus clusters, is learned automatically. By combining the ensemble methodology and nonparametric Bayesian modeling, NBCE addresses both the ill-posed nature and the parameter setting/model selection issues of clustering. Furthermore, NBCE outperforms individual clustering methods, since it can escape local optima by combining multiple clustering results. I also designed a nonparametric Bayesian co-clustering ensemble (NBCCE) technique. NBCCE inherits the advantages of NBCE, and in addition it is effective with high dimensional data. As such, NBCCE provides a unified framework to address all the three aforementioned issues. NBCCE assumes that multiple observed co-clustering results are generated from an unknown consensus co-clustering. The underlying distribution is assumed to be a mixture with a nonparametric Bayesian prior. I developed two models to generate co-clusters in terms of row- and column- clusters. In one case row- and column-clusters are assumed to be independent, and NBCCE assumes two independent Dirichlet Process priors on the hidden consensus co-clustering, one for rows and one for columns. The second model captures the dependence between row- and column-clusters by assuming a Mondrian Process prior on the hidden consensus co-clustering. Combined with Mondrian priors, NBCCE provides more flexibility to fit the data. I have performed extensive evaluation on relational data and protein-molecule interaction data. The empirical evaluation demonstrates the effectiveness of NBCE and NBCCE and their advantages over traditional clustering and co-clustering methods.

Download or read book Bayesian Decision Analysis written by Jim Q. Smith and published by Cambridge University Press. This book was released on 2010-09-23 with total page 349 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian decision analysis supports principled decision making in complex domains. This textbook takes the reader from a formal analysis of simple decision problems to a careful analysis of the sometimes very complex and data rich structures confronted by practitioners. The book contains basic material on subjective probability theory and multi-attribute utility theory, event and decision trees, Bayesian networks, influence diagrams and causal Bayesian networks. The author demonstrates when and how the theory can be successfully applied to a given decision problem, how data can be sampled and expert judgements elicited to support this analysis, and when and how an effective Bayesian decision analysis can be implemented. Evolving from a third-year undergraduate course taught by the author over many years, all of the material in this book will be accessible to a student who has completed introductory courses in probability and mathematical statistics.

Download or read book Bayesian Data Analysis Third Edition written by Andrew Gelman and published by CRC Press. This book was released on 2013-11-01 with total page 677 pages. Available in PDF, EPUB and Kindle. Book excerpt: Now in its third edition, this classic book is widely considered the leading text on Bayesian methods, lauded for its accessible, practical approach to analyzing data and solving research problems. Bayesian Data Analysis, Third Edition continues to take an applied approach to analysis using up-to-date Bayesian methods. The authors—all leaders in the statistics community—introduce basic concepts from a data-analytic perspective before presenting advanced methods. Throughout the text, numerous worked examples drawn from real applications and research emphasize the use of Bayesian inference in practice. New to the Third Edition Four new chapters on nonparametric modeling Coverage of weakly informative priors and boundary-avoiding priors Updated discussion of cross-validation and predictive information criteria Improved convergence monitoring and effective sample size calculations for iterative simulation Presentations of Hamiltonian Monte Carlo, variational Bayes, and expectation propagation New and revised software code The book can be used in three different ways. For undergraduate students, it introduces Bayesian inference starting from first principles. For graduate students, the text presents effective current approaches to Bayesian modeling and computation in statistics and related fields. For researchers, it provides an assortment of Bayesian methods in applied statistics. Additional materials, including data sets used in the examples, solutions to selected exercises, and software instructions, are available on the book’s web page.

Download or read book Bayesian Nonparametrics written by Nils Lid Hjort and published by Cambridge University Press. This book was released on 2010-04-12 with total page 309 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian nonparametrics works - theoretically, computationally. The theory provides highly flexible models whose complexity grows appropriately with the amount of data. Computational issues, though challenging, are no longer intractable. All that is needed is an entry point: this intelligent book is the perfect guide to what can seem a forbidding landscape. Tutorial chapters by Ghosal, Lijoi and Prünster, Teh and Jordan, and Dunson advance from theory, to basic models and hierarchical modeling, to applications and implementation, particularly in computer science and biostatistics. These are complemented by companion chapters by the editors and Griffin and Quintana, providing additional models, examining computational issues, identifying future growth areas, and giving links to related topics. This coherent text gives ready access both to underlying principles and to state-of-the-art practice. Specific examples are drawn from information retrieval, NLP, machine vision, computational biology, biostatistics, and bioinformatics.

Download or read book Bayesian Decision Analysis written by Jim Q. Smith and published by Cambridge University Press. This book was released on 2010-09-23 with total page 348 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian decision analysis supports principled decision making in complex domains. This textbook takes the reader from a formal analysis of simple decision problems to a careful analysis of the sometimes very complex and data rich structures confronted by practitioners. The book contains basic material on subjective probability theory and multi-attribute utility theory, event and decision trees, Bayesian networks, influence diagrams and causal Bayesian networks. The author demonstrates when and how the theory can be successfully applied to a given decision problem, how data can be sampled and expert judgements elicited to support this analysis, and when and how an effective Bayesian decision analysis can be implemented. Evolving from a third-year undergraduate course taught by the author over many years, all of the material in this book will be accessible to a student who has completed introductory courses in probability and mathematical statistics.

Download or read book Fundamentals of Nonparametric Bayesian Inference written by Subhashis Ghosal and published by Cambridge University Press. This book was released on 2017-06-26 with total page 671 pages. Available in PDF, EPUB and Kindle. Book excerpt: Explosive growth in computing power has made Bayesian methods for infinite-dimensional models - Bayesian nonparametrics - a nearly universal framework for inference, finding practical use in numerous subject areas. Written by leading researchers, this authoritative text draws on theoretical advances of the past twenty years to synthesize all aspects of Bayesian nonparametrics, from prior construction to computation and large sample behavior of posteriors. Because understanding the behavior of posteriors is critical to selecting priors that work, the large sample theory is developed systematically, illustrated by various examples of model and prior combinations. Precise sufficient conditions are given, with complete proofs, that ensure desirable posterior properties and behavior. Each chapter ends with historical notes and numerous exercises to deepen and consolidate the reader's understanding, making the book valuable for both graduate students and researchers in statistics and machine learning, as well as in application areas such as econometrics and biostatistics.

Download or read book Strategic Economic Decision Making written by Jeff Grover and published by Springer Science & Business Media. This book was released on 2012-12-05 with total page 122 pages. Available in PDF, EPUB and Kindle. Book excerpt: Strategic Economic Decision-Making: Using Bayesian Belief Networks to Solve Complex Problems is a quick primer on the topic that introduces readers to the basic complexities and nuances associated with learning Bayes’ theory and inverse probability for the first time. This brief is meant for non-statisticians who are unfamiliar with Bayes’ theorem, walking them through the theoretical phases of set and sample set selection, the axioms of probability, probability theory as it pertains to Bayes’ theorem, and posterior probabilities. All of these concepts are explained as they appear in the methodology of fitting a Bayes’ model, and upon completion of the text readers will be able to mathematically determine posterior probabilities of multiple independent nodes across any system available for study. Very little has been published in the area of discrete Bayes’ theory, and this brief will appeal to non-statisticians conducting research in the fields of engineering, computing, life sciences, and social sciences.

Download or read book Second Thoughts on the Bootstrap written by Bradley Efron and published by . This book was released on 2003 with total page 13 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Large Scale Inference written by Bradley Efron and published by Cambridge University Press. This book was released on 2012-11-29 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: We live in a new age for statistical inference, where modern scientific technology such as microarrays and fMRI machines routinely produce thousands and sometimes millions of parallel data sets, each with its own estimation or testing problem. Doing thousands of problems at once is more than repeated application of classical methods. Taking an empirical Bayes approach, Bradley Efron, inventor of the bootstrap, shows how information accrues across problems in a way that combines Bayesian and frequentist ideas. Estimation, testing and prediction blend in this framework, producing opportunities for new methodologies of increased power. New difficulties also arise, easily leading to flawed inferences. This book takes a careful look at both the promise and pitfalls of large-scale statistical inference, with particular attention to false discovery rates, the most successful of the new statistical techniques. Emphasis is on the inferential ideas underlying technical developments, illustrated using a large number of real examples.

Download or read book Bayesian Nonparametric Methods for Missing Data and Causal Inference written by Michael Joseph Daniels and published by Chapman & Hall/CRC. This book was released on 2024 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: "Bayesian nonparametric (BNP) methods can be used to flexibly model joint or conditional distributions, as well as functional relationships. These methods, along with causal and/or missingness assumptions, can be used with the g-formula to infer causal effects"--