Download or read book A Mixture based Framework for Nonparametric Density Estimation written by Chew-Seng Chee and published by . This book was released on 2011 with total page 142 pages. Available in PDF, EPUB and Kindle. Book excerpt: The primary goal of this thesis is to provide a mixture-based framework for nonparametric density estimation. This framework advocates the use of a mixture model with a nonparametric mixing distribution to approximate the distribution of the data. The implementation of a mixture-based nonparametric density estimator generally requires the specification of parameters in a mixture model and the choice of the bandwidth parameter. Consequently, a nonparametric methodology consisting of both the estimation and selection steps is described. For the estimation of parameters in mixture models, we employ the minimum disparity estimation framework within which there exist several estimation approaches differing in the way smoothing is incorporated in the disparity objective function. For the selection of the bandwidth parameter, we study some popular methods such as cross-validation and information criteria-based model selection methods. Also, new algorithms are developed for the computation of the mixture-based nonparametric density estimates. A series of studies on mixture-based nonparametric density estimators is presented, ranging from the problems of nonparametric density estimation in general to estimation under constraints. The problem of estimating symmetric densities is firstly investigated, followed by an extension in which the interest lies in estimating finite mixtures of symmetric densities. The third study utilizes the idea of double smoothing in defining the least squares criterion for mixture-based nonparametric density estimation. For these problems, numerical studies whether using both simulated and real data examples suggest that the performance of the mixture-based nonparametric density estimators is generally better than or at least competitive with that of the kernel-based nonparametric density estimators. The last but not least concern is nonparametric estimation of continuous and discrete distributions under shape constraints. Particularly, a new model called the discrete k-monotone is proposed for estimating the number of unknown species. In fact, the discrete k- monotone distribution is a mixture of specific discrete beta distributions. Empirica results indicate that the new model outperforms the commonly used nonparametric Poisson mixture model in the context of species richness estimation. Although there remain issues to be resolved, the promising results from our series of studies make the mixture-based framework a valuable tool for nonparametric density estimation.

Download or read book Likelihood based Density Estimation Using Deep Architectures written by Priyank Jaini and published by . This book was released on 2019 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Multivariate density estimation is a central problem in unsupervised machine learning that has been studied immensely in both statistics and machine learning. Several methods have thus been proposed for density estimation including classical techniques like histograms, kernel density estimation methods, mixture models, and more recently neural density estimation that leverages the recent advances in deep learning and neural networks to tractably represent a density function. In today's age, when large amounts of data are being generated in almost every field, it is of paramount importance to develop density estimation methods that are cheap both computationally and in memory cost. The main contribution of this thesis is in providing a principled study of parametric density estimation methods using mixture models and triangular maps for neural density estimation. The first part of the thesis focuses on the compact representation of mixture models using deep architectures like latent tree models, hidden Markov models, tensorial mixture models, hierarchical tensor formats and sum-product networks. It provides a unifying view of possible representations of mixture models using such deep architectures. The unifying view allows us to prove exponential separation between deep mixture models and mixture models represented using shallow architectures, demonstrating the benefits of depth in their representation. In a surprising result thereafter, we prove that a deep mixture model can be approximated using the conditional gradient algorithm by a shallow architecture of polynomial size w.r.t. the inverse of the approximation accuracy. Next, we address the more practical problem of density estimation of mixture models for streaming data by proposing an online Bayesian Moment Matching algorithm for Gaussian mixture models that can be distributed over several processors for fast computation. Exact Bayesian learning of mixture models is intractable because the number of terms in the posterior grows exponentially w.r.t. to the number of observations. We circumvent this problem by projecting the exact posterior on to a simple family of densities by matching a set of sufficient moments. Subsequently, we extend this algorithm for sequential data modeling using transfer learning by learning a hidden Markov model over the observations with Gaussian mixtures. We apply this algorithm on three diverse applications of activity recognition based on smartphone sensors, sleep stage classification for predicting neurological disorders using electroencephalography data and network size prediction for telecommunication networks. In the second part, we focus on neural density estimation methods where we provide a unified framework for estimating densities using monotone and bijective triangular maps represented using deep neural networks. Using this unified framework we study the limitations and representation power of recent flow based and autoregressive methods. Based on this framework, we subsequently propose a novel Sum-of-Squares polynomial flow that is interpretable, universal and easy to train.

Download or read book Nonparametric Model Selection written by Maurizio Tiso and published by . This book was released on 1999 with total page 372 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Nonparametric Function Estimation Modeling and Simulation written by James R. Thompson and published by SIAM. This book was released on 1990-01-01 with total page 320 pages. Available in PDF, EPUB and Kindle. Book excerpt: Topics emphasized include nonparametric density estimation as an exploratory device plus the deeper models to which the exploratory analysis points, multi-dimensional data analysis, and analysis of remote sensing data, cancer progression, chaos theory, epidemiological modeling, and parallel based algorithms. New methods discussed are quick nonparametric density estimation based techniques for resampling and simulation based estimation techniques not requiring closed form solutions.

Download or read book Models and Estimation Algorithms for Nonparametric Finite Mixtures with Conditionally Independent Multivariate Component Densities written by Vy-Thuy-Lynh Hoang and published by . This book was released on 2017 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Recently several authors have proposed models and estimation algorithms for finite nonparametric multivariate mixtures, whose identifiability is typically not obvious. Among the considered models, the assumption of independent coordinates conditional on the subpopulation from which each observation is drawn is subject of an increasing attention, in view of the theoretical and practical developments it allows, particularly with multiplicity of variables coming into play in the modern statistical framework. In this work we first consider a more general model assuming independence, conditional on the component, of multivariate blocks of coordinates instead of univariate coordinates, allowing for any dependence structure within these blocks. Consequently, the density functions of these blocks are completely multivariate and nonparametric. We present identifiability arguments and introduce for estimation in this model two methodological algorithms whose computational procedures resemble a true EM algorithm but include an additional density estimation step: a fast algorithm showing empirical efficiency without theoretical justification, and a smoothed algorithm possessing a monotony property as any EM algorithm does, but more computationally demanding. We also discuss computationally efficient methods for estimation and derive some strategies. Next, we consider a multivariate extension of the mixture models used in the framework of multiple hypothesis testings, allowing for a new multivariate version of the False Discovery Rate control. We propose a constrained version of our previous algorithm, specifically designed for this model. The behavior of the EM-type algorithms we propose is studied numerically through several Monte Carlo experiments and high dimensional real data, and compared with existing methods in the literature. Finally, the codes of our new algorithms are progressively implemented as new functions in the publicly-available package mixtools for the R statistical software.

Download or read book Nonparametric Density Estimation written by Luc Devroye and published by New York ; Toronto : Wiley. This book was released on 1985-01-18 with total page 376 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book gives a rigorous, systematic treatment of density estimates, their construction, use and analysis with full proofs. It develops L1 theory, rather than the classical L2, showing how L1 exposes fundamental properties of density estimates masked by L2.

Download or read book Finite Mixture Models written by Geoffrey McLachlan and published by John Wiley & Sons. This book was released on 2004-03-22 with total page 419 pages. Available in PDF, EPUB and Kindle. Book excerpt: An up-to-date, comprehensive account of major issues in finitemixture modeling This volume provides an up-to-date account of the theory andapplications of modeling via finite mixture distributions. With anemphasis on the applications of mixture models in both mainstreamanalysis and other areas such as unsupervised pattern recognition,speech recognition, and medical imaging, the book describes theformulations of the finite mixture approach, details itsmethodology, discusses aspects of its implementation, andillustrates its application in many common statisticalcontexts. Major issues discussed in this book include identifiabilityproblems, actual fitting of finite mixtures through use of the EMalgorithm, properties of the maximum likelihood estimators soobtained, assessment of the number of components to be used in themixture, and the applicability of asymptotic theory in providing abasis for the solutions to some of these problems. The author alsoconsiders how the EM algorithm can be scaled to handle the fittingof mixture models to very large databases, as in data miningapplications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and patternrecognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied andtheoretical statisticians as well as for researchers in the manyareas in which finite mixture models can be used to analyze data.

Download or read book Kernel based Nonparametric Density Estimation and Regression with Statistical Applications written by Peter John Foster and published by . This book was released on 1990 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Python Data Science Handbook written by Jake VanderPlas and published by "O'Reilly Media, Inc.". This book was released on 2016-11-21 with total page 743 pages. Available in PDF, EPUB and Kindle. Book excerpt: For many researchers, Python is a first-class tool mainly because of its libraries for storing, manipulating, and gaining insight from data. Several resources exist for individual pieces of this data science stack, but only with the Python Data Science Handbook do you get them all—IPython, NumPy, Pandas, Matplotlib, Scikit-Learn, and other related tools. Working scientists and data crunchers familiar with reading and writing Python code will find this comprehensive desk reference ideal for tackling day-to-day issues: manipulating, transforming, and cleaning data; visualizing different types of data; and using data to build statistical or machine learning models. Quite simply, this is the must-have reference for scientific computing in Python. With this handbook, you’ll learn how to use: IPython and Jupyter: provide computational environments for data scientists using Python NumPy: includes the ndarray for efficient storage and manipulation of dense data arrays in Python Pandas: features the DataFrame for efficient storage and manipulation of labeled/columnar data in Python Matplotlib: includes capabilities for a flexible range of data visualizations in Python Scikit-Learn: for efficient and clean Python implementations of the most important and established machine learning algorithms

Download or read book Segmentation and Recovery of Superquadrics written by Ales Jaklic and published by Springer Science & Business Media. This book was released on 2000-09-30 with total page 318 pages. Available in PDF, EPUB and Kindle. Book excerpt: A representation of objects by their parts is the dominant strategy for representing complex 3D objects in many disciplines. In computer vision and robotics, superquadrics are among the most widespread part models. Superquadrics are a family of parametric models that cover a wide variety of smoothly changing 3D symmetric shapes, which are controlled with a small number of parameters and which can be augmented with the addition of global and local deformations. The book covers, in depth, the geometric properties of superquadrics. The main contribution of the book is an original approach to the recovery and segmentation of superquadrics from range images. Several applications of superquadrics in computer vision and robotics are thoroughly discussed and, in particular, the use of superquadrics for range image registration is demonstrated. Audience: The book is intended for readers of all levels who are familiar with and interested in computer vision issues.

Download or read book Flexible Bayesian Regression Modelling written by Yanan Fan and published by Academic Press. This book was released on 2019-10-30 with total page 302 pages. Available in PDF, EPUB and Kindle. Book excerpt: Flexible Bayesian Regression Modeling is a step-by-step guide to the Bayesian revolution in regression modeling, for use in advanced econometric and statistical analysis where datasets are characterized by complexity, multiplicity, and large sample sizes, necessitating the need for considerable flexibility in modeling techniques. It reviews three forms of flexibility: methods which provide flexibility in their error distribution; methods which model non-central parts of the distribution (such as quantile regression); and finally models that allow the mean function to be flexible (such as spline models). Each chapter discusses the key aspects of fitting a regression model. R programs accompany the methods. This book is particularly relevant to non-specialist practitioners with intermediate mathematical training seeking to apply Bayesian approaches in economics, biology, finance, engineering and medicine. Introduces powerful new nonparametric Bayesian regression techniques to classically trained practitioners Focuses on approaches offering both superior power and methodological flexibility Supplemented with instructive and relevant R programs within the text Covers linear regression, nonlinear regression and quantile regression techniques Provides diverse disciplinary case studies for correlation and optimization problems drawn from Bayesian analysis ‘in the wild’

Download or read book Nonparametric Estimation of Multivariate Elliptic Densities Via Finite Mixture Sieves written by Heather Battey and published by . This book was released on 2013 with total page 69 pages. Available in PDF, EPUB and Kindle. Book excerpt: This paper considers the class of p-dimensional elliptic distributions (p ≥ 1) satisfying the consistency property (Kano, 1994) and within this general framework presents a two-stage semiparametric estimator for the Lebesgue density based on Gaussian mixture sieves. Under the on-line Exponentiated Gradient (EG) algorithm of Helmbold et al. (1997) and without restricting the mixing measure to have compact support, the estimator produces estimates converging uniformly in probability to the true elliptic density at a rate that is independent of the dimension of the problem, hence circumventing the familiar curse of dimensionality inherent to many semiparametric estimators. The rate performance of our estimator depends on the tail behaviour of the underlying mixing density (and hence that of the data) rather than smoothness properties. In fact, our method achieves a rate of at least Op(n-1/4), provided only some positive moment exists. When further moments exists, the rate improves reaching Op(n-3/8) as the tails of the true density converge to those of a normal. Unlike the elliptic density estimator of Liebscher (2005), our sieve estimator always yields an estimate that is a valid density, and is also attractive from a practical perspective as it accepts data as a stream, thus significantly reducing computational and storage requirements.Monte Carlo experimentation indicates encouraging finite sample performance over a range of elliptic densities. The estimator is also implemented in a binary classification task using the well-known Wisconsin breast cancer dataset.

Download or read book Combinatorial Methods in Density Estimation written by Luc Devroye and published by Springer Science & Business Media. This book was released on 2012-12-06 with total page 219 pages. Available in PDF, EPUB and Kindle. Book excerpt: Density estimation has evolved enormously since the days of bar plots and histograms, but researchers and users are still struggling with the problem of the selection of the bin widths. This book is the first to explore a new paradigm for the data-based or automatic selection of the free parameters of density estimates in general so that the expected error is within a given constant multiple of the best possible error. The paradigm can be used in nearly all density estimates and for most model selection problems, both parametric and nonparametric.

Download or read book Nonparametric Density Estimation Using Correlated Data with Application to Screening Problems written by Junjun Qin and published by . This book was released on 2013 with total page 141 pages. Available in PDF, EPUB and Kindle. Book excerpt: To detect a rare phenomenon with a higher probability by means of some other variable(s) which is (are) highly related but easier in technology or cheaper in expenditure is termed as "screening". It has come to be realized in recent years that current methodologies are of limited practical value. They are either built on a parametric framework which in fact we have little knowledge about or little involved with "multivariate screening" and "multi-stage screening", which are more frequent and more important in practice. Therefore, the alternative method discussed here suggests that we consider a nonparametric density estimation using correlated data. This one derives from and greatly formulates a rough idea that screening in nature is a kind of problems about how to solve probabilities and thus we intuitively look for estimating the probability density function (pdf). Since we have little information about data in a regular setting, nonparametric estimation seems a best candidate. In this study, the approach is presented by firstly making nonparametric density estimation in the case where random variables X and Y are both continuous and in the case where X is continuous but Y is discrete respectively. The behaviors under the large sample setting are explored and the performance of the proposed method is evaluated through simulations. The proposed method is then applied to a subsequent screening problem. The comparison with some current method is also presented.

Download or read book On Statistical Pattern Recognition in Independent Component Analysis Mixture Modelling written by Addisson Salazar and published by Springer Science & Business Media. This book was released on 2012-07-20 with total page 200 pages. Available in PDF, EPUB and Kindle. Book excerpt: A natural evolution of statistical signal processing, in connection with the progressive increase in computational power, has been exploiting higher-order information. Thus, high-order spectral analysis and nonlinear adaptive filtering have received the attention of many researchers. One of the most successful techniques for non-linear processing of data with complex non-Gaussian distributions is the independent component analysis mixture modelling (ICAMM). This thesis defines a novel formalism for pattern recognition and classification based on ICAMM, which unifies a certain number of pattern recognition tasks allowing generalization. The versatile and powerful framework developed in this work can deal with data obtained from quite different areas, such as image processing, impact-echo testing, cultural heritage, hypnograms analysis, web-mining and might therefore be employed to solve many different real-world problems.

Download or read book Nonparametric Density Estimation for Bounded Data written by Song Li and published by . This book was released on 2015 with total page 217 pages. Available in PDF, EPUB and Kindle. Book excerpt: This thesis proposes nonparametric methods for the estimation of univariate density in a unit interval, as well as bivariate copula density. A unified Bayesian framework is proposed for choosing the optimal kernel function and bandwidth. The results of the simulation studies establish the finite sample performances of proposed methods relative to existing counterparts. Relative to existing methods, the proposed methods are shown to adequately estimate the univariate probability density of recovery rates of financially distressed firms and bond issuers. Additionally, the proposed copula has superior tail risk forecasts of All Ordinaries index returns conditional on S&P 500 index returns.

Download or read book Nonparametric Density Estimation with Applications written by Steven J. Bean and published by . This book was released on 1979 with total page 316 pages. Available in PDF, EPUB and Kindle. Book excerpt: