Download or read book Almost Nonparametric and Nonparametric Estimation in Mixture Models written by and published by . This book was released on 2001 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Nonparametric Estimation in Multivariate Finite Mixture Models written by Tatiana A. Benaglia and published by . This book was released on 2008 with total page 129 pages. Available in PDF, EPUB and Kindle. Book excerpt: The main goal of this thesis is to provide a complete methodology to analyze finite multivariate mixture models. Our approach is fully nonparametric and it only requires the coordinates to be independent, conditional on the component membership. Since the number of components in the mixture is assumed to be known, we also developed a technique to select the number of components that best fits the model. In addition, we provide some tools to check if the assumption of conditional independence is reasonable. All the methods are evaluated by simulations, and compared with methodology existing in the literature. We also apply our methodology to two real datasets from cognitive psychology.
Download or read book Nonparametric Statistics and Mixture Models written by David R. Hunter and published by World Scientific. This book was released on 2011 with total page 370 pages. Available in PDF, EPUB and Kindle. Book excerpt: This festschrift includes papers authored by many collaborators, colleagues, and students of Professor Thomas P Hettmansperger, who worked in research in nonparametric statistics, rank statistics, robustness, and mixture models during a career that spanned nearly 40 years. It is a broad sample of peer-reviewed, cutting-edge research related to nonparametrics and mixture models.
Download or read book Nonparametric Statistics And Mixture Models A Festschrift In Honor Of Thomas P Hettmansperger written by David Hunter and published by World Scientific. This book was released on 2011-01-03 with total page 370 pages. Available in PDF, EPUB and Kindle. Book excerpt: This festschrift includes papers authored by many collaborators, colleagues, and students of Professor Thomas P Hettmansperger, who worked in research in nonparametric statistics, rank statistics, robustness, and mixture models during a career that spanned nearly 40 years. It is a broad sample of peer-reviewed, cutting-edge research related to nonparametrics and mixture models.
Download or read book A Mixture based Framework for Nonparametric Density Estimation written by Chew-Seng Chee and published by . This book was released on 2011 with total page 142 pages. Available in PDF, EPUB and Kindle. Book excerpt: The primary goal of this thesis is to provide a mixture-based framework for nonparametric density estimation. This framework advocates the use of a mixture model with a nonparametric mixing distribution to approximate the distribution of the data. The implementation of a mixture-based nonparametric density estimator generally requires the specification of parameters in a mixture model and the choice of the bandwidth parameter. Consequently, a nonparametric methodology consisting of both the estimation and selection steps is described. For the estimation of parameters in mixture models, we employ the minimum disparity estimation framework within which there exist several estimation approaches differing in the way smoothing is incorporated in the disparity objective function. For the selection of the bandwidth parameter, we study some popular methods such as cross-validation and information criteria-based model selection methods. Also, new algorithms are developed for the computation of the mixture-based nonparametric density estimates. A series of studies on mixture-based nonparametric density estimators is presented, ranging from the problems of nonparametric density estimation in general to estimation under constraints. The problem of estimating symmetric densities is firstly investigated, followed by an extension in which the interest lies in estimating finite mixtures of symmetric densities. The third study utilizes the idea of double smoothing in defining the least squares criterion for mixture-based nonparametric density estimation. For these problems, numerical studies whether using both simulated and real data examples suggest that the performance of the mixture-based nonparametric density estimators is generally better than or at least competitive with that of the kernel-based nonparametric density estimators. The last but not least concern is nonparametric estimation of continuous and discrete distributions under shape constraints. Particularly, a new model called the discrete k-monotone is proposed for estimating the number of unknown species. In fact, the discrete k- monotone distribution is a mixture of specific discrete beta distributions. Empirica results indicate that the new model outperforms the commonly used nonparametric Poisson mixture model in the context of species richness estimation. Although there remain issues to be resolved, the promising results from our series of studies make the mixture-based framework a valuable tool for nonparametric density estimation.
Download or read book Non Parametric Finite Multivariate Mixture Models with Applications written by XIAOTIAN ZHU and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This research set out to investigate and build upon the foundation for the nonparametric estimation of finite multivariate mixture models given the conditional independence assumption, set forth in a series of studies over the last decade. We proposed a novel formulation of the objective function in terms of penalized smoothed Kullback-Leibler divergence under a reduced parameter space. A special optimization landscape and scheme was discovered in working out the majorizationminimization method for the estimation problem which leads to a closed form of the nonlinearly smoothed majorization-minimization (NSMM) algorithm. We established a sharpened monotonicity property that precisely measures the distance between successive iterates of the algorithm and proved the existence of a solution to the main optimization problem for the first time in literature. The estimation theory for this basic model together with the special optimization scheme can be adapted to the investigation of an important extension of the model that incorporates component-wise independent component analysis (ICA). The NSMMICA algorithm has been developed and a discretized version of it, which interweaves NSMM and weighted FastICA has been implemented in the R package icamix as a model-based clustering tool. We demonstrated the use of the newly developed methods/algorithms by applications in image analysis and unsupervised learning.
Download or read book Finite Mixture Models written by Geoffrey McLachlan and published by John Wiley & Sons. This book was released on 2004-03-22 with total page 419 pages. Available in PDF, EPUB and Kindle. Book excerpt: An up-to-date, comprehensive account of major issues in finitemixture modeling This volume provides an up-to-date account of the theory andapplications of modeling via finite mixture distributions. With anemphasis on the applications of mixture models in both mainstreamanalysis and other areas such as unsupervised pattern recognition,speech recognition, and medical imaging, the book describes theformulations of the finite mixture approach, details itsmethodology, discusses aspects of its implementation, andillustrates its application in many common statisticalcontexts. Major issues discussed in this book include identifiabilityproblems, actual fitting of finite mixtures through use of the EMalgorithm, properties of the maximum likelihood estimators soobtained, assessment of the number of components to be used in themixture, and the applicability of asymptotic theory in providing abasis for the solutions to some of these problems. The author alsoconsiders how the EM algorithm can be scaled to handle the fittingof mixture models to very large databases, as in data miningapplications. This comprehensive, practical guide: * Provides more than 800 references-40% published since 1995 * Includes an appendix listing available mixture software * Links statistical literature with machine learning and patternrecognition literature * Contains more than 100 helpful graphs, charts, and tables Finite Mixture Models is an important resource for both applied andtheoretical statisticians as well as for researchers in the manyareas in which finite mixture models can be used to analyze data.
Download or read book Models and Estimation Algorithms for Nonparametric Finite Mixtures with Conditionally Independent Multivariate Component Densities written by Vy-Thuy-Lynh Hoang and published by . This book was released on 2017 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Recently several authors have proposed models and estimation algorithms for finite nonparametric multivariate mixtures, whose identifiability is typically not obvious. Among the considered models, the assumption of independent coordinates conditional on the subpopulation from which each observation is drawn is subject of an increasing attention, in view of the theoretical and practical developments it allows, particularly with multiplicity of variables coming into play in the modern statistical framework. In this work we first consider a more general model assuming independence, conditional on the component, of multivariate blocks of coordinates instead of univariate coordinates, allowing for any dependence structure within these blocks. Consequently, the density functions of these blocks are completely multivariate and nonparametric. We present identifiability arguments and introduce for estimation in this model two methodological algorithms whose computational procedures resemble a true EM algorithm but include an additional density estimation step: a fast algorithm showing empirical efficiency without theoretical justification, and a smoothed algorithm possessing a monotony property as any EM algorithm does, but more computationally demanding. We also discuss computationally efficient methods for estimation and derive some strategies. Next, we consider a multivariate extension of the mixture models used in the framework of multiple hypothesis testings, allowing for a new multivariate version of the False Discovery Rate control. We propose a constrained version of our previous algorithm, specifically designed for this model. The behavior of the EM-type algorithms we propose is studied numerically through several Monte Carlo experiments and high dimensional real data, and compared with existing methods in the literature. Finally, the codes of our new algorithms are progressively implemented as new functions in the publicly-available package mixtools for the R statistical software.
Download or read book Nonparametric Estimation of the Mixing Distribution in Mixed Models with Random Intercepts and Slopes written by Rabih Saab and published by . This book was released on 2013 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Generalized linear mixture models (GLMM) are widely used in statistical applications to model count and binary data. We consider the problem of nonparametric likelihood estimation of mixing distributions in GLMM's with multiple random effects. The log-likelihood to be maximized has the general form l(G)=?i log?f(yi,?) dG(?)where f(.,?) is a parametric family of component densities, yi is the ith observed response dependent variable, and G is a mixing distribution function of the random effects vector ? defined on ?.The literature presents many algorithms for maximum likelihood estimation (MLE) of G in the univariate random effect case such as the EM algorithm (Laird, 1978), the intra-simplex direction method, ISDM (Lesperance and Kalbfleish, 1992), and vertex exchange method, VEM (Bohning, 1985). In this dissertation, the constrained Newton method (CNM) in Wang (2007), which fits GLMM's with random intercepts only, is extended to fit clustered datasets with multiple random effects. Owing to the general equivalence theorem from the geometry of mixture likelihoods (see Lindsay, 1995), many NPMLE algorithms including CNM and ISDM maximize the directional derivative of the log-likelihood to add potential support points to the mixing distribution G. Our method, Direct Search Directional Derivative (DSDD), uses a directional search method to find local maxima of the multi-dimensional directional derivative function. The DSDD's performance is investigated in GLMM where f is a Bernoulli or Poisson distribution function. The algorithm is also extended to cover GLMM's with zero-inflated data. Goodness-of-fit (GOF) and selection methods for mixed models have been developed in the literature, however their application in models with nonparametric random effects distributions is vague and ad-hoc. Some popular measures such as the Deviance Information Criteria (DIC), conditional Akaike Information Criteria (cAIC) and R2 statistics are potentially useful in this context. Additionally, some cross-validation goodness-of-fit methods popular in Bayesian applications, such as the conditional predictive ordinate (CPO) and numerical posterior predictive checks, can be applied with some minor modifications to suit the non-Bayesian approach.
Download or read book Estimation in Semiparametric Models written by Johann Pfanzagl and published by Springer Science & Business Media. This book was released on 2012-12-06 with total page 116 pages. Available in PDF, EPUB and Kindle. Book excerpt: Assume one has to estimate the mean J x P( dx) (or the median of P, or any other functional t;;(P)) on the basis ofi.i.d. observations from P. Ifnothing is known about P, then the sample mean is certainly the best estimator one can think of. If P is known to be the member of a certain parametric family, say {Po: {) E e}, one can usually do better by estimating {) first, say by {)(n)(.~.), and using J XPo(n)(;r.) (dx) as an estimate for J xPo(dx). There is an "intermediate" range, where we know something about the unknown probability measure P, but less than parametric theory takes for granted. Practical problems have always led statisticians to invent estimators for such intermediate models, but it usually remained open whether these estimators are nearly optimal or not. There was one exception: The case of "adaptivity", where a "nonparametric" estimate exists which is asymptotically optimal for any parametric submodel. The standard (and for a long time only) example of such a fortunate situation was the estimation of the center of symmetry for a distribution of unknown shape.
Download or read book Introduction to Nonparametric Estimation written by Alexandre B. Tsybakov and published by Springer Science & Business Media. This book was released on 2008-10-22 with total page 222 pages. Available in PDF, EPUB and Kindle. Book excerpt: Developed from lecture notes and ready to be used for a course on the graduate level, this concise text aims to introduce the fundamental concepts of nonparametric estimation theory while maintaining the exposition suitable for a first approach in the field.
Download or read book A Conditional Gradient Approach for Nonparametric Estimation of Mixing Distributions written by Srikanth Jagabathula and published by . This book was released on 2020 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models are versatile tools that are used extensively in many fields, including operations, marketing, and econometrics. The main challenge in estimating mixture models is that the mixing distribution is often unknown and imposing apriori parametric assumptions can lead to model misspecification issues. In this paper, we propose a new methodology for nonparametric estimation of the mixing distribution of a mixture of logit models. We formulate the likelihood-based estimation problem as a constrained convex program and apply the conditional gradient (a.k.a. Frank-Wolfe) algorithm to solve this convex program. We show that our method iteratively generates the support of the mixing distribution and the mixing proportions. Theoretically, we establish sublinear convergence rate of our estimator and characterize the structure of the recovered mixing distribution. Empirically, we test our approach on real-world datasets. We show that it outperforms the standard expectation-maximization (EM) benchmark on speed (16x faster), in-sample fit (up to 24% reduction in the log-likelihood loss), and predictive (average 27% reduction in standard error metrics) and decision accuracies (extracts around 23% more revenue). On synthetic data, we show that our estimator is robust to different ground-truth mixing distributions and can also account for endogeneity.
Download or read book Nonparametric Techniques in Finite Mixture of Regression Models written by Mian Huang and published by . This book was released on 2009 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models have been popular in the literature of both statistics and social science. In this dissertation, we propose a new mixture model, namely, nonparametric finite mixture of regression models, which can be viewed as a natural extension of finite mixture of linear regression. In the newly proposed model, it allows both the regression and variance function as functions of covariates, and their functional forms are nonparametric rather than a specified form. We first consider the mixing proportion in the nonparametric finite mixture of regression models is also a nonparametric function of covariates. We develop an estimation procedure for the nonparametric finite mixture of regression models by employing kernel regression, and proposed an algorithm to carry out the estimation procedure by modifying an EM algorithm. We further systematically studied the sampling properties of the newly proposed estimation procedures and the proposed algorithm. We found that the proposed algorithm preserves the ascent property of the EM algorithm in an asymptotic sense. We derive the asymptotic bias and variance of the resulting estimate. We further established the asymptotic normality of the resulting estimate. Monte Carlo simulation studies are conducted to assess the finite sample performance of the resulting estimate. The proposed methodology is illustrated by analysis of a real data example. We further study the nonparametric finite mixture of regression models with constant mixing proportion. Since the mixing proportion is parametric, while the regression function and variance function for each components are nonparametric, the model indeed is a semiparametric model. To achieve better convergent rate for mixing proportional parameters, we develop an estimation procedures by using back-fitting algorithm. To reduce computational cost, we further suggest one-step back-fitting algorithm, which behaves similar to the gradient ECM algorithm. Thus, the convergence behavior of the proposed algorithm can be analyzed along the lines for the gradient EM algorithm. We studied the asymptotic properties of the resulting estimate. We showed that the resulting estimate for the mixing proportion parameter is root $n$ consistent, and follows an asymptotic normal distribution. We also derived the asymptotic bias and variance for the resulting estimate of the regression function and variance function, and further established their asymptotic normality. Finite sample performance of the proposed procedure is examined by a Monte Carlo simulation study. The proposed procedure is demonstrated by analysis of a real data example. As the advent of data collection technology and data storage device, researchers are able to collect functional data without much cost. In this dissertation, we studied mixture models for functional data. More specifically, we proposed mixtures of Gaussian processes for functional data. The proposed model is a natural extension of mixture of high-dimensional normals. We develop an estimation procedure to the mean and covariance function of mixture of Gaussian processes by using kernel regression. The proposed methodology is empirically justified by simulation and illustrated by an analysis of the supermarket data.
Download or read book Estimation of Finite Mixture Models written by and published by . This book was released on 2004 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: A recorded signal frequently results from the mixture of many signals from several classifiable sources. Knowledge of the contribution of the underlying sources to the recorded signal is valuable in several applications, such as remote sensing. Such mixtures may be analyzed using finite mixture models. Historically, finite mixture models decompose a density as the sum of a finite number of component densities. Current methods for estimating the contribution of each component assume a parametric form for the mixture components. Furthermore, these methods assume a collection of samples from the mixture are observed rather than an aggregate representation of the samples, such as a histogram. This work introduces a method to address the many practical cases where parametric mixture models are insufficient to describe the mixture components. The observed mixture is assumed to occur in an aggregate representation of samples. Thus, the mixture components are represented as finite-length signals or vectors. The proposed method incorporates the first and second order statistics of the mixture components obtained from previously collected samples of the mixture components. The new method is based on the set theoretic method of successive projections onto convex sets (POCS). The set theoretic approach defines a set of feasible solutions as the intersection of sets consistent with the prior knowledge of a desirable solution. POCS is an iterative procedure used to find a point in the set of feasible solutions. This work considers several sets describing the finite mixture model, including a new model set generalizing a set based on the error-in-variables model. To illustrate the viability of the new method, comparisons are made with the expectation-maximization (EM) algorithm for mixtures with parametric components. Simulations of mixture with nonparametric components emphasize the advantages of the new method, since no other methods address mixtures with nonparametric component.
Download or read book Finite Mixture and Markov Switching Models written by Sylvia Frühwirth-Schnatter and published by Springer Science & Business Media. This book was released on 2006-11-24 with total page 506 pages. Available in PDF, EPUB and Kindle. Book excerpt: The past decade has seen powerful new computational tools for modeling which combine a Bayesian approach with recent Monte simulation techniques based on Markov chains. This book is the first to offer a systematic presentation of the Bayesian perspective of finite mixture modelling. The book is designed to show finite mixture and Markov switching models are formulated, what structures they imply on the data, their potential uses, and how they are estimated. Presenting its concepts informally without sacrificing mathematical correctness, it will serve a wide readership including statisticians as well as biologists, economists, engineers, financial and market researchers.
Download or read book Handbook of Mixture Analysis written by Sylvia Fruhwirth-Schnatter and published by CRC Press. This book was released on 2019-01-04 with total page 388 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixture models have been around for over 150 years, and they are found in many branches of statistical modelling, as a versatile and multifaceted tool. They can be applied to a wide range of data: univariate or multivariate, continuous or categorical, cross-sectional, time series, networks, and much more. Mixture analysis is a very active research topic in statistics and machine learning, with new developments in methodology and applications taking place all the time. The Handbook of Mixture Analysis is a very timely publication, presenting a broad overview of the methods and applications of this important field of research. It covers a wide array of topics, including the EM algorithm, Bayesian mixture models, model-based clustering, high-dimensional data, hidden Markov models, and applications in finance, genomics, and astronomy. Features: Provides a comprehensive overview of the methods and applications of mixture modelling and analysis Divided into three parts: Foundations and Methods; Mixture Modelling and Extensions; and Selected Applications Contains many worked examples using real data, together with computational implementation, to illustrate the methods described Includes contributions from the leading researchers in the field The Handbook of Mixture Analysis is targeted at graduate students and young researchers new to the field. It will also be an important reference for anyone working in this field, whether they are developing new methodology, or applying the models to real scientific problems.
Download or read book Statistical Analysis of Finite Mixture Distributions written by D. M. Titterington and published by . This book was released on 1985 with total page 264 pages. Available in PDF, EPUB and Kindle. Book excerpt: In this book, the authors give a complete account of the applications, mathematical structure and statistical analysis of finite mixture distributions.
