Download or read book Nonparametric Bayesian Models for Unsupervised Learning written by Pu Wang and published by . This book was released on 2011 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Unsupervised learning is an important topic in machine learning. In particular, clustering is an unsupervised learning problem that arises in a variety of applications for data analysis and mining. Unfortunately, clustering is an ill-posed problem and, as such, a challenging one: no ground-truth that can be used to validate clustering results is available. Two issues arise as a consequence. Various clustering algorithms embed their own bias resulting from different optimization criteria. As a result, each algorithm may discover different patterns in a given dataset. The second issue concerns the setting of parameters. In clustering, parameter setting controls the characterization of individual clusters, and the total number of clusters in the data. Clustering ensembles have been proposed to address the issue of different biases induced by various algorithms. Clustering ensembles combine different clustering results, and can provide solutions that are robust against spurious elements in the data. Although clustering ensembles provide a significant advance, they do not address satisfactorily the model selection and the parameter tuning problem. Bayesian approaches have been applied to clustering to address the parameter tuning and model selection issues. Bayesian methods provide a principled way to address these problems by assuming prior distributions on model parameters. Prior distributions assign low probabilities to parameter values which are unlikely. Therefore they serve as regularizers for modeling parameters, and can help avoid over-fitting. In addition, the marginal likelihood is used by Bayesian approaches as the criterion for model selection. Although Bayesian methods provide a principled way to perform parameter tuning and model selection, the key question \How many clusters?" is still open. This is a fundamental question for model selection. A special kind of Bayesian methods, nonparametric Bayesian approaches, have been proposed to address this important model selection issue. Unlike parametric Bayesian models, for which the number of parameters is finite and fixed, nonparametric Bayesian models allow the number of parameters to grow with the number of observations. After observing the data, nonparametric Bayesian models t the data with finite dimensional parameters. An additional issue with clustering is high dimensionality. High-dimensional data pose a difficult challenge to the clustering process. A common scenario with high-dimensional data is that clusters may exist in different subspaces comprised of different combinations of features (dimensions). In other words, data points in a cluster may be similar to each other along a subset of dimensions, but not in all dimensions. People have proposed subspace clustering techniques, a.k.a. co-clustering or bi-clustering, to address the dimensionality issue (here, I use the term co-clustering). Like clustering, also co-clustering suffers from the ill-posed nature and the lack of ground-truth to validate the results. Although attempts have been made in the literature to address individually the major issues related to clustering, no previous work has addressed them jointly. In my dissertation I propose a unified framework that addresses all three issues at the same time. I designed a nonparametric Bayesian clustering ensemble (NBCE) approach, which assumes that multiple observed clustering results are generated from an unknown consensus clustering. The under- lying distribution is assumed to be a mixture distribution with a nonparametric Bayesian prior, i.e., a Dirichlet Process. The number of mixture components, a.k.a. the number of consensus clusters, is learned automatically. By combining the ensemble methodology and nonparametric Bayesian modeling, NBCE addresses both the ill-posed nature and the parameter setting/model selection issues of clustering. Furthermore, NBCE outperforms individual clustering methods, since it can escape local optima by combining multiple clustering results. I also designed a nonparametric Bayesian co-clustering ensemble (NBCCE) technique. NBCCE inherits the advantages of NBCE, and in addition it is effective with high dimensional data. As such, NBCCE provides a unified framework to address all the three aforementioned issues. NBCCE assumes that multiple observed co-clustering results are generated from an unknown consensus co-clustering. The underlying distribution is assumed to be a mixture with a nonparametric Bayesian prior. I developed two models to generate co-clusters in terms of row- and column- clusters. In one case row- and column-clusters are assumed to be independent, and NBCCE assumes two independent Dirichlet Process priors on the hidden consensus co-clustering, one for rows and one for columns. The second model captures the dependence between row- and column-clusters by assuming a Mondrian Process prior on the hidden consensus co-clustering. Combined with Mondrian priors, NBCCE provides more flexibility to fit the data. I have performed extensive evaluation on relational data and protein-molecule interaction data. The empirical evaluation demonstrates the effectiveness of NBCE and NBCCE and their advantages over traditional clustering and co-clustering methods.
Download or read book Bayesian Nonparametrics written by J.K. Ghosh and published by Springer Science & Business Media. This book was released on 2006-05-11 with total page 311 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book is the first systematic treatment of Bayesian nonparametric methods and the theory behind them. It will also appeal to statisticians in general. The book is primarily aimed at graduate students and can be used as the text for a graduate course in Bayesian non-parametrics.
Download or read book Nonparametric Bayesian Methods for Supervised and Unsupervised Learning written by Vikash Kumar Mansinghka and published by . This book was released on 2009 with total page 90 pages. Available in PDF, EPUB and Kindle. Book excerpt: I introduce two nonparametric Bayesian methods for solving problems of supervised and unsupervised learning. The first method simultaneously learns causal networks and causal theories from data. For example, given synthetic co-occurrence data from a simple causal model for the medical domain, it can learn relationships like "having a flu causes coughing", while also learning that observable quantities can be usefully grouped into categories like diseases and symptoms, and that diseases tend to cause symptoms, not the other way around. The second method is an online algorithm for learning a prototype-based model for categorial concepts, and can be used to solve problems of multiclass classification with missing features. I apply it to problems of categorizing newsgroup posts and recognizing handwritten digits. These approaches were inspired by a striking capacity of human learning, which should also be a desideratum for any intelligent system: the ability to learn certain kinds of "simple" or "natural" structures very quickly, while still being able to learn arbitrary -- and arbitrarily complex - structures given enough data. In each case, I show how nonparametric Bayesian modeling and inference based on stochastic simulation give us some of the tools we need to achieve this goal.
Download or read book Nonparametric Bayesian Discrete Latent Variable Models for Unsupervised Learning written by and published by . This book was released on 2007 with total page 134 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Nonparametric Bayesian Models for Machine Learning written by Romain Jean Thibaux and published by . This book was released on 2008 with total page 150 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Bayesian Nonparametrics via Neural Networks written by Herbert K. H. Lee and published by SIAM. This book was released on 2004-01-01 with total page 106 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian Nonparametrics via Neural Networks is the first book to focus on neural networks in the context of nonparametric regression and classification, working within the Bayesian paradigm. Its goal is to demystify neural networks, putting them firmly in a statistical context rather than treating them as a black box. This approach is in contrast to existing books, which tend to treat neural networks as a machine learning algorithm instead of a statistical model. Once this underlying statistical model is recognized, other standard statistical techniques can be applied to improve the model. The Bayesian approach allows better accounting for uncertainty. This book covers uncertainty in model choice and methods to deal with this issue, exploring a number of ideas from statistics and machine learning. A detailed discussion on the choice of prior and new noninformative priors is included, along with a substantial literature review. Written for statisticians using statistical terminology, Bayesian Nonparametrics via Neural Networks will lead statisticians to an increased understanding of the neural network model and its applicability to real-world problems.
Download or read book Nonparametric Bayesian Learning for Collaborative Robot Multimodal Introspection written by Xuefeng Zhou and published by Springer Nature. This book was released on 2020-01-01 with total page 149 pages. Available in PDF, EPUB and Kindle. Book excerpt: This open access book focuses on robot introspection, which has a direct impact on physical human-robot interaction and long-term autonomy, and which can benefit from autonomous anomaly monitoring and diagnosis, as well as anomaly recovery strategies. In robotics, the ability to reason, solve their own anomalies and proactively enrich owned knowledge is a direct way to improve autonomous behaviors. To this end, the authors start by considering the underlying pattern of multimodal observation during robot manipulation, which can effectively be modeled as a parametric hidden Markov model (HMM). They then adopt a nonparametric Bayesian approach in defining a prior using the hierarchical Dirichlet process (HDP) on the standard HMM parameters, known as the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM). The HDP-HMM can examine an HMM with an unbounded number of possible states and allows flexibility in the complexity of the learned model and the development of reliable and scalable variational inference methods. This book is a valuable reference resource for researchers and designers in the field of robot learning and multimodal perception, as well as for senior undergraduate and graduate university students.
Download or read book Bayesian Nonparametrics written by Nils Lid Hjort and published by Cambridge University Press. This book was released on 2010-04-12 with total page 309 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian nonparametrics works - theoretically, computationally. The theory provides highly flexible models whose complexity grows appropriately with the amount of data. Computational issues, though challenging, are no longer intractable. All that is needed is an entry point: this intelligent book is the perfect guide to what can seem a forbidding landscape. Tutorial chapters by Ghosal, Lijoi and Prünster, Teh and Jordan, and Dunson advance from theory, to basic models and hierarchical modeling, to applications and implementation, particularly in computer science and biostatistics. These are complemented by companion chapters by the editors and Griffin and Quintana, providing additional models, examining computational issues, identifying future growth areas, and giving links to related topics. This coherent text gives ready access both to underlying principles and to state-of-the-art practice. Specific examples are drawn from information retrieval, NLP, machine vision, computational biology, biostatistics, and bioinformatics.
Download or read book Bayesian Nonparametric Data Analysis written by Peter Müller and published by Springer. This book was released on 2015-06-17 with total page 203 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book reviews nonparametric Bayesian methods and models that have proven useful in the context of data analysis. Rather than providing an encyclopedic review of probability models, the book’s structure follows a data analysis perspective. As such, the chapters are organized by traditional data analysis problems. In selecting specific nonparametric models, simpler and more traditional models are favored over specialized ones. The discussed methods are illustrated with a wealth of examples, including applications ranging from stylized examples to case studies from recent literature. The book also includes an extensive discussion of computational methods and details on their implementation. R code for many examples is included in online software pages.
Download or read book Fundamentals of Nonparametric Bayesian Inference written by Subhashis Ghosal and published by Cambridge University Press. This book was released on 2017-06-26 with total page 671 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian nonparametrics comes of age with this landmark text synthesizing theory, methodology and computation.
Download or read book Bayesian Reasoning and Machine Learning written by David Barber and published by Cambridge University Press. This book was released on 2012-02-02 with total page 739 pages. Available in PDF, EPUB and Kindle. Book excerpt: A practical introduction perfect for final-year undergraduate and graduate students without a solid background in linear algebra and calculus.
Download or read book Prior Processes and Their Applications written by Eswar G. Phadia and published by Springer. This book was released on 2016-07-27 with total page 337 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book presents a systematic and comprehensive treatment of various prior processes that have been developed over the past four decades for dealing with Bayesian approach to solving selected nonparametric inference problems. This revised edition has been substantially expanded to reflect the current interest in this area. After an overview of different prior processes, it examines the now pre-eminent Dirichlet process and its variants including hierarchical processes, then addresses new processes such as dependent Dirichlet, local Dirichlet, time-varying and spatial processes, all of which exploit the countable mixture representation of the Dirichlet process. It subsequently discusses various neutral to right type processes, including gamma and extended gamma, beta and beta-Stacy processes, and then describes the Chinese Restaurant, Indian Buffet and infinite gamma-Poisson processes, which prove to be very useful in areas such as machine learning, information retrieval and featural modeling. Tailfree and Polya tree and their extensions form a separate chapter, while the last two chapters present the Bayesian solutions to certain estimation problems pertaining to the distribution function and its functional based on complete data as well as right censored data. Because of the conjugacy property of some of these processes, most solutions are presented in closed form. However, the current interest in modeling and treating large-scale and complex data also poses a problem – the posterior distribution, which is essential to Bayesian analysis, is invariably not in a closed form, making it necessary to resort to simulation. Accordingly, the book also introduces several computational procedures, such as the Gibbs sampler, Blocked Gibbs sampler and slice sampling, highlighting essential steps of algorithms while discussing specific models. In addition, it features crucial steps of proofs and derivations, explains the relationships between different processes and provides further clarifications to promote a deeper understanding. Lastly, it includes a comprehensive list of references, equipping readers to explore further on their own.
Download or read book A Nonparametric Bayesian Perspective for Machine Learning in Partially observed Settings written by Ferit Akova and published by . This book was released on 2013 with total page 184 pages. Available in PDF, EPUB and Kindle. Book excerpt: Robustness and generalizability of supervised learning algorithms depend on the quality of the labeled data set in representing the real-life problem. In many real-world domains, however, we may not have full knowledge of the underlying data-generating mechanism, which may even have an evolving nature introducing new classes continually. This constitutes a partially-observed setting, where it would be impractical to obtain a labeled data set exhaustively defined by a fixed set of classes. Traditional supervised learning algorithms, assuming an exhaustive training library, would misclassify a future sample of an unobserved class with probability one, leading to an ill-defined classification problem. Our goal is to address situations where such assumption is violated by a non-exhaustive training library, which is a very realistic yet an overlooked issue in supervised learning. In this dissertation we pursue a new direction for supervised learning by defining self-adjusting models to relax the fixed model assumption imposed on classes and their distributions. We let the model adapt itself to the prospective data by dynamically adding new classes/components as data demand, which in turn gradually make the model more representative of the entire population. In this framework, we first employ suitably chosen nonparametric priors to model class distributions for observed as well as unobserved classes and then, utilize new inference methods to classify samples from observed classes and discover/model novel classes for those from unobserved classes. This thesis presents the initiating steps of an ongoing effort to address one of the most overlooked bottlenecks in supervised learning and indicates the potential for taking new perspectives in some of the most heavily studied areas of machine learning: novelty detection, online class discovery and semi-supervised learning.
Download or read book Bayesian Methods for Nonlinear Classification and Regression written by David G. T. Denison and published by John Wiley & Sons. This book was released on 2002-05-06 with total page 302 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bei der Regressionsanalyse von Datenmaterial erhält man leider selten lineare oder andere einfache Zusammenhänge (parametrische Modelle). Dieses Buch hilft Ihnen, auch komplexere, nichtparametrische Modelle zu verstehen und zu beherrschen. Stärken und Schwächen jedes einzelnen Modells werden durch die Anwendung auf Standarddatensätze demonstriert. Verbreitete nichtparametrische Modelle werden mit Hilfe von Bayes-Verfahren in einen kohärenten wahrscheinlichkeitstheoretischen Zusammenhang gebracht.
Download or read book Nonparametric Bayesian Modelling in Machine Learning written by Nada Habli and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Bayesian Nonparametric Methods for Non exchangeable Data written by Nicholas J. Foti and published by . This book was released on 2013 with total page 358 pages. Available in PDF, EPUB and Kindle. Book excerpt: Bayesian nonparametric methods have become increasingly popular in machine learning for their ability to allow the data to determine model complexity. In particular, Bayesian nonparametric versions of common latent variable models can learn as effective dimension of the latent space. Examples include mixture models, latent feature models and topic models, where the number of components, features, or topics need not be specified a priori. A drawback of many of these models is that they assume the observations are exchangeable, that is, any dependencies between observations are ignored. This thesis contributes general methods to incorporate covariates into Bayesian nonparametric models and inference algorithms to learn with these models. First, we will present a flexible class of dependent Bayesian nonparametric priors to induce covariate-dependence into a variety of latent variable models used in machine learning. The proposed framework has nice analytic properites and admits a simple inference algorithm. We show how the framework can be used to construct a covariate-dependent latent feature model and a time-varying topic model. Second, we describe the first general purpose inference algorithm for a large family of dependent mixture models. Using the idea of slice-sampling, the algorithm is truncation-free and fast, showing that inference can de done efficiently despite the added complexity that covariate-dependence entails. Last, we construct a Bayesian nonparametric framework to couple multiple latent variable models and apply the framework to learning from multiple views of data.
Download or read book Nonparametric Bayesian Methods for Extracting Structure from Data written by Edward Meeds and published by . This book was released on with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
