Download or read book Adaptive Model Selection in Linear Mixed Models written by Bo Zhang and published by . This book was released on 2009 with total page 148 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book ADAPTIVE LASSO FOR MIXED MODEL SELECTION VIA PROFILE LOG LIKELIHOOD written by Juming Pan and published by . This book was released on 2016 with total page 140 pages. Available in PDF, EPUB and Kindle. Book excerpt: Linear mixed models describe the relationship between a response variable and some predictors for data that are grouped according to one or more clustering factors. A linear mixed model consists of both fixed effects and random effects. Fixed effects are the conventional linear regression coefficients, and random effects are associated with units which are drawn randomly from a population. By accommodating such two types of parameters, linear mixed models provide an effective and flexible way of representing the means as well as the covariance structure of the data, therefore have been primarily used to model correlated data, and have received much attention in a variety of disciplines including agriculture, biology, medicine, and sociology. Due to the complex nature of the linear mixed models, the selection of only important covariates to create an interpretable model becomes challenging as the dimension of fixed or random effects increases. Thus, determining an appropriate structural form for a model to be used in making inferences and predictions is a fundamental problem in the analysis of longitudinal or clustered data using linear mixed models. This dissertation focuses on selection and estimation for linear mixed models by integrating the recent advances in model selection. More specifically, we propose a two-stage penalized procedure for selecting and estimating important fixed and random effects. Compared with the traditional subset selection approaches, penalized methods can enhance the predictive power of a model, and can significantly reduce computational cost when the number of variables is large (Fan and Li, 2001). Our proposed procedure is different from the existing ones in the literature mainly in two aspects. First, the proposed method is composed of two stages to separately choose the parameters of interests, therefore can respect and accommodate the distinct properties between the random and fixed effects. Second, the usage of the profile log-likelihoods in the selection process can make the computation more efficient and stable due to a smaller number of dimensions involved. In the first stage, we choose the random effects by maximizing the penalized restricted profile log-likelihood, and the maximization is completed by the Newton-Raphson algorithm. Observe that if a random effect is a noise variable, then the corresponding variance components should be all zero. Thus, we first estimate the covariance matrix of random effects using the adaptive LASSO penalized method and then identify the vital ones based on the estimated covariance matrix. In the view of such a selection procedure, the selected random effects are invariant to the selection of the fixed effects. When a proper model for the covariance is adopted, the correct covariance structure will be obtained and valid inferences for the fixed effects can then be achieved in the next stage. We further study the theoretical properties of the proposed procedure for random effects selection. We prove that, with probability tending to one, the proposed procedure surely identifies all true random effects. After the completion of the random effects selection, in the second stage, we select the fixed effects through the maximization of the penalized profile log-likelihood, which only involves the regression coefficients. The optimization of the penalized profile log-likelihood can be solved by the Newton-Raphson algorithm. We then investigate the sampling properties of the resulting estimate of fixed effects. We show that the resulting estimate enjoys model selection oracle properties, indicating that asymptotically the proposed approach can discover the subset of significant predictors. After finishing the two-stage penalized procedure, the best linear mixed model can subsequently be determined and be applied to handle correlated data in a number of fields. To illustrate the performance of the proposed method, numerous simulation studies have been conducted. The simulation results demonstrate that the proposed technique is quite efficient in selecting the best covariates and random covariance structure in linear mixed models and outperforms the existing selection methodologies in general. We finally apply the method to two real data applications for further examining its effectiveness in mixed model selection.

Download or read book Adaptive Model Selection written by Yongli Zhang and published by . This book was released on 2007 with total page 174 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book The Fence Methods written by Jiming Jiang and published by World Scientific. This book was released on 2015-09-22 with total page 248 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book is about a recently developed class of strategies, known as the fence methods, which fits particularly well in non-conventional and complex model selection problems with practical considerations. The idea involves a procedure to isolate a subgroup of what are known as correct models, of which the optimal model is a member. This is accomplished by constructing a statistical fence, or barrier, to carefully eliminate incorrect models. Once the fence is constructed, the optimal model is selected from amongst those within the fence according to a criterion which can be made flexible. In particular, the criterion of optimality can incorporate consideration of practical interest, thus making model selection a real life practice.Furthermore, this book introduces a data-driven approach, called adaptive fence, which can be used in a wide range of problems involving determination of tuning parameters, or constants. Instead of relying on asymptotic theory, the fence focuses on finite-sample performance, and computation. Such features are particularly suitable to statistics in the new era.

Download or read book Study on Adaptive Model Selection Through Generalized Degrees of Freedom in Nested Linear Regression Models written by 湯泉發 and published by . This book was released on 2010 with total page 84 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Robust Mixed Model Analysis written by Jiang Jiming and published by World Scientific. This book was released on 2019-04-10 with total page 268 pages. Available in PDF, EPUB and Kindle. Book excerpt: Mixed-effects models have found broad applications in various fields. As a result, the interest in learning and using these models is rapidly growing. On the other hand, some of these models, such as the linear mixed models and generalized linear mixed models, are highly parametric, involving distributional assumptions that may not be satisfied in real-life problems. Therefore, it is important, from a practical standpoint, that the methods of inference about these models are robust to violation of model assumptions. Fortunately, there is a full scale of methods currently available that are robust in certain aspects. Learning about these methods is essential for the practice of mixed-effects models.This research monograph provides a comprehensive account of methods of mixed model analysis that are robust in various aspects, such as to violation of model assumptions, or to outliers. It is suitable as a reference book for a practitioner who uses the mixed-effects models, and a researcher who studies these models. It can also be treated as a graduate text for a course on mixed-effects models and their applications.

Download or read book Mixed Effects Models for the Population Approach written by Marc Lavielle and published by CRC Press. This book was released on 2014-07-14 with total page 385 pages. Available in PDF, EPUB and Kindle. Book excerpt: Wide-Ranging Coverage of Parametric Modeling in Linear and Nonlinear Mixed Effects Models Mixed Effects Models for the Population Approach: Models, Tasks, Methods and Tools presents a rigorous framework for describing, implementing, and using mixed effects models. With these models, readers can perform parameter estimation and modeling across a whole population of individuals at the same time. Easy-to-Use Techniques and Tools for Real-World Data Modeling The book first shows how the framework allows model representation for different data types, including continuous, categorical, count, and time-to-event data. This leads to the use of generic methods, such as the stochastic approximation of the EM algorithm (SAEM), for modeling these diverse data types. The book also covers other essential methods, including Markov chain Monte Carlo (MCMC) and importance sampling techniques. The author uses publicly available software tools to illustrate modeling tasks. Methods are implemented in Monolix, and models are visually explored using Mlxplore and simulated using Simulx. Careful Balance of Mathematical Representation and Practical Implementation This book takes readers through the whole modeling process, from defining/creating a parametric model to performing tasks on the model using various mathematical methods. Statisticians and mathematicians will appreciate the rigorous representation of the models and theoretical properties of the methods while modelers will welcome the practical capabilities of the tools. The book is also useful for training and teaching in any field where population modeling occurs.

Download or read book A PENALIZED APPROACH TO MIXED MODEL SELECTION VIA CROSS VALIDATION written by Jingwei Xiong and published by . This book was released on 2017 with total page 95 pages. Available in PDF, EPUB and Kindle. Book excerpt: A linear mixed model is a useful technique to explain observations by regarding them as realizations of random variables, especially when repeated measurements are made to statistical units, such as longitudinal data. However, in practice, there are often too many potential factors considered affecting the observations, while actually, they are not. Therefore, statisticians have been trying to select significant factors out of all the potential factors, where we call the process model selection. Among those approaches for linear mixed model selection, penalized methods have been developed profoundly over the last several decades. In this dissertation, to solve the overfitting problem in most penalized methods and improve the selection accuracy, we mainly focus on a penalized approach via cross-validation. Unlike the existing methods using the whole data to fit and select models, we split the fitting process and selection into two stages. More specifically, an adaptive lasso penalized function is customized in the first stage and marginal BIC criterion is used in the second stage. We consider that the main advantage of our approach is to reduce the dependency between models construction and evaluation. Because of the complex structure of mixed models, we adopt a modified Cholesky decomposition to reparameterize the model, which in turn significantly reduces the dimension of the penalized function. Additionally, since random effects are missing, there is no closed form for the maximizer of the penalized function, thus we implement EM algorithm to obtain a full inference of parameters. Furthermore, due to the computation limit and moderately small samples in practice, some noisy factors may still remain in the model, which is particularly obvious for fixed effects. To eliminate the noisy factors, a likelihood ratio test is employed to screen the fixed effects. Regarding the overall process, we call it adaptive lasso via cross-validation. Additionally, we demonstrate that the proposed approach possesses selection and estimation consistency simultaneously. Moreover, simulation studies and real data examples are both provided to justify the method validity. At the very end, a brief conclusion is drawn and some possible further improvements are discussed.

Download or read book Model Selection in Linear Mixed effects Models written by Long Huu Ngo and published by . This book was released on 2000 with total page 320 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Generalized Linear Mixed Models written by Charles E. McCulloch and published by IMS. This book was released on 2003 with total page 100 pages. Available in PDF, EPUB and Kindle. Book excerpt: Wiley Series in Probability and Statistics A modern perspective on mixed models The availability of powerful computing methods in recent decades has thrust linear and nonlinear mixed models into the mainstream of statistical application. This volume offers a modern perspective on generalized, linear, and mixed models, presenting a unified and accessible treatment of the newest statistical methods for analyzing correlated, nonnormally distributed data. As a follow-up to Searle's classic, Linear Models, and Variance Components by Searle, Casella, and McCulloch, this new work progresses from the basic one-way classification to generalized linear mixed models. A variety of statistical methods are explained and illustrated, with an emphasis on maximum likelihood and restricted maximum likelihood. An invaluable resource for applied statisticians and industrial practitioners, as well as students interested in the latest results, Generalized, Linear, and Mixed Models features: * A review of the basics of linear models and linear mixed models * Descriptions of models for nonnormal data, including generalized linear and nonlinear models * Analysis and illustration of techniques for a variety of real data sets * Information on the accommodation of longitudinal data using these models * Coverage of the prediction of realized values of random effects * A discussion of the impact of computing issues on mixed models

Download or read book Adaptive Regression for Modeling Nonlinear Relationships written by George J. Knafl and published by Springer. This book was released on 2016-09-20 with total page 384 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book presents methods for investigating whether relationships are linear or nonlinear and for adaptively fitting appropriate models when they are nonlinear. Data analysts will learn how to incorporate nonlinearity in one or more predictor variables into regression models for different types of outcome variables. Such nonlinear dependence is often not considered in applied research, yet nonlinear relationships are common and so need to be addressed. A standard linear analysis can produce misleading conclusions, while a nonlinear analysis can provide novel insights into data, not otherwise possible. A variety of examples of the benefits of modeling nonlinear relationships are presented throughout the book. Methods are covered using what are called fractional polynomials based on real-valued power transformations of primary predictor variables combined with model selection based on likelihood cross-validation. The book covers how to formulate and conduct such adaptive fractional polynomial modeling in the standard, logistic, and Poisson regression contexts with continuous, discrete, and counts outcomes, respectively, either univariate or multivariate. The book also provides a comparison of adaptive modeling to generalized additive modeling (GAM) and multiple adaptive regression splines (MARS) for univariate outcomes. The authors have created customized SAS macros for use in conducting adaptive regression modeling. These macros and code for conducting the analyses discussed in the book are available through the first author's website and online via the book’s Springer website. Detailed descriptions of how to use these macros and interpret their output appear throughout the book. These methods can be implemented using other programs.

Download or read book Robust Adaptive Scheme for Linear Mixed Model written by Gabriel Asare Okyere and published by . This book was released on 2011 with total page 216 pages. Available in PDF, EPUB and Kindle. Book excerpt: If the underlying distribution of a statistical model is known, then a procedure which maximizes power and efficiency can be selected. For example, if the distribution of errors is known to be normal in a linear model, then inference based on least squares maximizes power and efficiency. More generally, if this distribution is known, then a rank-based inference based on the appropriate rank score function has maximum efficiency. In practice, though, this distribution is not known. Adaptive schemes are procedures which hopefully select appropriate methods to optimize the analysis. Hogg (1974) presented an adaptive rank-based scheme for testing in the simple two-sample location model. It consists of a family of distribution free tests, each associated with a rank score function, and a selection procedure which chooses a test from this family. For continuous error distributions, Hogg's scheme is valid for testing; that is, it retains the level. This, however, is not true for estimation and fitting. Little has been done to extend Hogg's scheme beyond simple location problems. In this research we extend Hogg's adaptive scheme for testing to mixed models consisting of m clusters of observations. This is an important practical family of models, including, for example, repeated measure designs, multi-center clinical designs, and randomized block designs. Under the assumption of exchangeable errors, we establish that our testing scheme is valid. In practice, though, for these models fitting is crucial. Based on the fitting of a model, a residual analysis can be performed to check, say, quality of fit and to determine outliers. Further, standard errors of the estimates can then be obtained so that confidence intervals for contrasts of interests can readily be formed. With this in mind, we have also developed several adaptive fitting schemes for these models. These are based on several types of rank-based estimation procedures, including joint ranking and multiple ranking types of fitting. Our schemes are robust and higly efficient. A large Monte Carlo study over error distributions ranging from heavy-tailed to light-tailed distributions and from symmetric distributions to skewed distributions gives empirical credence to our adaptive procedures. We illustrate our procedures with several real clinical examples.

Download or read book Linear and Generalized Linear Mixed Models and Their Applications written by Jiming Jiang and published by Springer Nature. This book was released on 2021-03-22 with total page 343 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book covers two major classes of mixed effects models, linear mixed models and generalized linear mixed models. It presents an up-to-date account of theory and methods in analysis of these models as well as their applications in various fields. The book offers a systematic approach to inference about non-Gaussian linear mixed models. Furthermore, it includes recently developed methods, such as mixed model diagnostics, mixed model selection, and jackknife method in the context of mixed models. The book is aimed at students, researchers and other practitioners who are interested in using mixed models for statistical data analysis.

Download or read book Mixed Effects Models for Complex Data written by Lang Wu and published by CRC Press. This book was released on 2009-11-11 with total page 431 pages. Available in PDF, EPUB and Kindle. Book excerpt: Although standard mixed effects models are useful in a range of studies, other approaches must often be used in correlation with them when studying complex or incomplete data. Mixed Effects Models for Complex Data discusses commonly used mixed effects models and presents appropriate approaches to address dropouts, missing data, measurement errors, censoring, and outliers. For each class of mixed effects model, the author reviews the corresponding class of regression model for cross-sectional data. An overview of general models and methods, along with motivating examples After presenting real data examples and outlining general approaches to the analysis of longitudinal/clustered data and incomplete data, the book introduces linear mixed effects (LME) models, generalized linear mixed models (GLMMs), nonlinear mixed effects (NLME) models, and semiparametric and nonparametric mixed effects models. It also includes general approaches for the analysis of complex data with missing values, measurement errors, censoring, and outliers. Self-contained coverage of specific topics Subsequent chapters delve more deeply into missing data problems, covariate measurement errors, and censored responses in mixed effects models. Focusing on incomplete data, the book also covers survival and frailty models, joint models of survival and longitudinal data, robust methods for mixed effects models, marginal generalized estimating equation (GEE) models for longitudinal or clustered data, and Bayesian methods for mixed effects models. Background material In the appendix, the author provides background information, such as likelihood theory, the Gibbs sampler, rejection and importance sampling methods, numerical integration methods, optimization methods, bootstrap, and matrix algebra. Failure to properly address missing data, measurement errors, and other issues in statistical analyses can lead to severely biased or misleading results. This book explores the biases that arise when naïve methods are used and shows which approaches should be used to achieve accurate results in longitudinal data analysis.

Download or read book Shrinkage Parameter Selection in Generalized Linear and Mixed Models written by Erin K. Melcon and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Penalized likelihood methods such as lasso, adaptive lasso, and SCAD have been highly utilized in linear models. Selection of the penalty parameter is an important step in modeling with penalized techniques. Traditionally, information criteria or cross validation are used to select the penalty parameter. Although methods of selecting this have been evaluated in linear models, general linear models and linear mixed models have not been so thoroughly explored.This dissertation will introduce a data-driven bootstrap (Empirical Optimal Selection, or EOS) approach for selecting the penalty parameter with a focus on model selection. We implement EOS on selecting the penalty parameter in the case of lasso and adaptive lasso. In generalized linear models we will introduce the method, show simulations comparing EOS to information criteria and cross validation, and give theoretical justification for this approach. We also consider a practical upper bound for the penalty parameter, with theoretical justification. In linear mixed models, we use EOS with two different objective functions; the traditional log-likelihood approach (which requires an EM algorithm), and a predictive approach. In both of these cases, we compare selecting the penalty parameter with EOS to selection with information criteria. Theoretical justification for both objective functions and a practical upper bound for the penalty parameter in the log-likelihood case are given. We also applied our technique to two datasets; the South African heart data (logistic regression) and the Yale infant data (a linear mixed model). For the South African data, we compare the final models using EOS and information criteria via the mean squared prediction error (MSPE). For the Yale infant data, we compare our results to those obtained by Ibrahim et al. (2011).

Download or read book Linear Mixed Models written by Brady T. West and published by CRC Press. This book was released on 2006-11-22 with total page 376 pages. Available in PDF, EPUB and Kindle. Book excerpt: Simplifying the often confusing array of software programs for fitting linear mixed models (LMMs), Linear Mixed Models: A Practical Guide Using Statistical Software provides a basic introduction to primary concepts, notation, software implementation, model interpretation, and visualization of clustered and longitudinal data. This easy-to-nav

Download or read book Essays on Robust Model Selection and Model Averaging for Linear Models written by Le Chang and published by . This book was released on 2017 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Model selection is central to all applied statistical work. Selecting the variables for use in a regression model is one important example of model selection. This thesis is a collection of essays on robust model selection procedures and model averaging for linear regression models. In the first essay, we propose robust Akaike information criteria (AIC) for MM-estimation and an adjusted robust scale based AIC for M and MM-estimation. Our proposed model selection criteria can maintain their robust properties in the presence of a high proportion of outliers and the outliers in the covariates. We compare our proposed criteria with other robust model selection criteria discussed in previous literature. Our simulation studies demonstrate a significant outperformance of robust AIC based on MM-estimation in the presence of outliers in the covariates. The real data example also shows a better performance of robust AIC based on MM-estimation. The second essay focuses on robust versions of the "Least Absolute Shrinkage and Selection Operator" (lasso). The adaptive lasso is a method for performing simultaneous parameter estimation and variable selection. The adaptive weights used in its penalty term mean that the adaptive lasso achieves the oracle property. In this essay, we propose an extension of the adaptive lasso named the Tukey-lasso. By using Tukey's biweight criterion, instead of squared loss, the Tukey-lasso is resistant to outliers in both the response and covariates. Importantly, we demonstrate that the Tukey-lasso also enjoys the oracle property. A fast accelerated proximal gradient (APG) algorithm is proposed and implemented for computing the Tukey-lasso. Our extensive simulations show that the Tukey-lasso, implemented with the APG algorithm, achieves very reliable results, including for high-dimensional data where p>n. In the presence of outliers, the Tukey-lasso is shown to offer substantial improvements in performance compared to the adaptive lasso and other robust implementations of the lasso. Real data examples further demonstrate the utility of the Tukey-lasso. In many statistical analyses, a single model is used for statistical inference, ignoring the process that leads to the model being selected. To account for this model uncertainty, many model averaging procedures have been proposed. In the last essay, we propose an extension of a bootstrap model averaging approach, called bootstrap lasso averaging (BLA). BLA utilizes the lasso for model selection. This is in contrast to other forms of bootstrap model averaging that use AIC or Bayesian information criteria (BIC). The use of the lasso improves the computation speed and allows BLA to be applied even when the number of variables p is larger than the sample size n. Extensive simulations confirm that BLA has outstanding finite sample performance, in terms of both variable and prediction accuracies, compared with traditional model selection and model averaging methods. Several real data examples further demonstrate an improved out-of-sample predictive performance of BLA.