Download or read book Multivariate Adaptive Regression Spline Based Framework for Statistically Parsimonious Adaptive Dyanmic Programming written by Subrat Sahu and published by . This book was released on 2011 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Central to Dynamic Programming (DP) is the 'cost-to-go' or 'future value' function, which is obtained via solving the Bellman's equation, and central to many Approximate Dynamic Programming (ADP) methods is the approximation of the future value function. The exact DP algorithm seeks to compute and store a table consisting of one cost-to-go value for each point in the state space, and its usefulness is limited by the curse of dimensionality, which renders the methodology computationally intractable in the face of real life problems with high-dimensional state spaces and in the face of continuous state variables. ADP methodology seeks to address and redress the issue of the curse of dimensionality by not seeking to compute the future value function exactly and at each point of the state space; rather opting for an approximation of the future value function in the domain of the state space. Existing ADP methodologies have successfully handled 'continuous' state variables through discretization of the state space and estimation of the cost-to-go or future value function and have been challenged in scenarios involving a mix of 'continuous' and 'categorical' or qualitative state variables. The first part of this dissertation research seeks to develop a flexible, nonparametric statistical modeling method which can capture complex nonlinearity in data comprised of a mix of continuous and categorical variables and can be used to approximate future value functions in stochastic dynamic programming (SDP) problems with a mix of continuous and categorical state variables. This dissertation proposes a statistical modeling method, called 'TreeMARS' which combines the versatility of tree-models with the flexibility of multivariate adaptive regression splines (MARS). An extension of the proposed model, called 'Boosted TreeMARS', is also presented. Comparisons are made to the tree-regression model that uses a similar concept, but only permits the use of linear regression at the terminal nodes. Comparisons are presented on a 10-dimensional simulated data set. The second part of the dissertation research, seeking statistical parsimony, proposes a sequential statistical modeling methodology utilizing the 'sequential' concept from Design and Analysis of Computer Experiments (DACE) to make the grid 'only fine enough' for the 'efficient' discretization and then use MARS methods to approximate future value functions. This methodology can be extended in the future to use tree-based MARS models to approximate future value functions involving a mix of continuous and categorical state variables. This sequential grid discretization is nothing but sequential exploration of the state space and this concept of sequential exploration of the state space provides a statistically parsimonious ADP methodology which 'adaptively' captures the important variables from the state space and builds approximations around these quantities while seeking approximation of the future value functions, by using adaptive and flexible modeling.

Download or read book Variants of Multivariate Adaptive Regression Splines MARS written by Diana Luisa Martinez Cepeda and published by . This book was released on 2013 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Multivariate Adaptive Regression Splines (MARS) is a statistical modeling method used to represent high-dimensional data with interactions. It uses different algorithms to select the terms to be included in the approximation model that best represent the data. In addition, it performs a variable selection, therefore the most significant predictors are shown in the final model. Design and analysis of computer experiments (DACE) is a statistical technique for creating approximations (called metamodels) of computer models. For optimization problems in which there is an unknown function that must be approximated, DACE approach could be applied. In stochastic dynamic programming (SDP) for example, a metamodel can be used to approximate the unknown future value function.The goal of DACE is to efficiently predict the response value of a computer model. MARS has been used as a metamodel in DACE technique. MARS is a flexible model, however in optimization, certain characteristics may be desired, such as a convex or piecewise-linear structure. To satisfy these characteristics, different variants of MARS have been developed. By enabling these variants, MARS modeling facilitates the optimization process. These variations include the ability to model a convex function, a piecewise-linear function and to provide a smoothing option using a quintic routine.DACE has had an enormous contribution for studying complex system, however one of consistent concerns for the researchers is computational time. As researchers seek to study more and more complex systems, corresponding computer models continue to push the limits of computing power. To overcome this drawback, efficient sequential approaches have been studied to reduce the computational effort. This research work focuses its efforts on the development of sequential approaches based on MARS model. The objective is to sequentially update the approximation function using current and new input data points. Additionally, by using less input data points, an accurate prediction of the unknown function could be obtained in a faster manner, and thus the complexity of the model structure is less. This could also facilitate the optimization process.Different case studies are shown in order to test the different MARS variants and sequential MARS approaches proposed in this dissertation. These cases include an inventory forecasting problem, an automotive crash safety design problem and an air pollution SDP problem.

Download or read book Convex Versions of Multivariate Adaptive Regression Splines and Implementations for Complex Optimization Problems written by Dachuan Thomas Shih and published by . This book was released on 2006 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Multivariate Adaptive Regression Splines (MARS) provide a flexible statistical modeling method that employs forward and backward search algorithms to identify the combination of basis functions that best fits the data. In optimization, MARS has been used successfully to estimate the value function in stochastic dynamic programming, and MARS could be potentially useful in many real world optimization problems where objective (or other) functions need to be estimated from data, such as in simulation optimization. Many optimization methods depend on convexity, but a nonconvex MARS approximation is inherently possible because interaction terms are products of univariate terms. In this dissertation, convex versions of MARS are proposed. In order to ensure MARS convexity, two major modifications are made: (1) coefficients are constrained such that pairs of basis functions are guaranteed to jointly form convex functions; (2) The form of interaction terms is appropriately changed. Finally, MARS convexity can be achieved by the fact that the sum of convex functions is convex. The implementation of MARS for approximating complex optimization functions can involve hundreds to thousands of state or decision variables. In particular, this research studies application to an inventory forecasting stochastic dynamic programming problem and an airline fleet assignment problem. Although one can simply attempt a MARS approximation over all the variables, prior research on the fleet assignment application indicates that many variables have little effect on the objective. Thus, a data mining step to conduct variable selection is needed. This step separates potentially critical variables from clearly redundant ones. In this dissertation, variants of two data mining tools are explored separately and in combination for variable selection: regression trees and multiple testing procedures based on false discovery rate.

Download or read book Adaptive Regression for Modeling Nonlinear Relationships written by George J. Knafl and published by Springer. This book was released on 2016-09-20 with total page 384 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book presents methods for investigating whether relationships are linear or nonlinear and for adaptively fitting appropriate models when they are nonlinear. Data analysts will learn how to incorporate nonlinearity in one or more predictor variables into regression models for different types of outcome variables. Such nonlinear dependence is often not considered in applied research, yet nonlinear relationships are common and so need to be addressed. A standard linear analysis can produce misleading conclusions, while a nonlinear analysis can provide novel insights into data, not otherwise possible. A variety of examples of the benefits of modeling nonlinear relationships are presented throughout the book. Methods are covered using what are called fractional polynomials based on real-valued power transformations of primary predictor variables combined with model selection based on likelihood cross-validation. The book covers how to formulate and conduct such adaptive fractional polynomial modeling in the standard, logistic, and Poisson regression contexts with continuous, discrete, and counts outcomes, respectively, either univariate or multivariate. The book also provides a comparison of adaptive modeling to generalized additive modeling (GAM) and multiple adaptive regression splines (MARS) for univariate outcomes. The authors have created customized SAS macros for use in conducting adaptive regression modeling. These macros and code for conducting the analyses discussed in the book are available through the first author's website and online via the book’s Springer website. Detailed descriptions of how to use these macros and interpret their output appear throughout the book. These methods can be implemented using other programs.

Download or read book Multivariate Adaptive Regression Splines written by Stanford University. Dept. of Statistics. Laboratory for Computational Statistics and published by . This book was released on 1990 with total page 156 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book An Investigation of Multivariate Adaptive Regression Splines for Modeling and Analysis of Univariate and Semi multivariate Time Series Systems written by James G. Stevens and published by . This book was released on 1991 with total page 192 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Combining Multivariate Adaptive Regression Splines with a Response Surface Methodology for Simulation based Design Optimization written by Scott T. Crino and published by . This book was released on 2006 with total page 374 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book G SPLINES a Hybrid of Friedman s Multivariate Adaptive Regression Splines MARS Algorithm with Holland s Genetic Algorithm written by Research Institute for Advanced Computer Science (U.S.) and published by . This book was released on 1991 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Nonlinear Modeling of Time Series Using Multivariate Adaptive Regression Splines MARS written by Peter A. W. Lewis and published by . This book was released on 1990 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: MARS(Multivariate Adaptive Regression Splines). Abstract: MARS is a new methodology, due to Friedman, for nonlinear regression modeling. MARS can be conceptualized as a generalization of recursive partitioning that uses spline fitting in lieu of other simple functions. Given a set of predictor variables, MARS fits a model in a form of an expansion of product spline basis functions of predictors chosen during a forward and backward recursive partitioning strategy. MARS produces continuous models for discrete data that can have multiple partitions and multilinear terms. Predictor variable contributions and interactions in a MARS model may be analyzed using an ANOVA style decomposition. By letting the predictor variables in MARS be lagged values of a time series, one obtains a new method for nonlinear autoregressive threshold modeling of time series. A significant feature of this extension of MARS is its ability to produce models with limit cycles when modeling time series data that exhibit periodic behavior. In a physical context, limit cycles represent a stationary state of sustained oscillations, a satisfying behavior for any model of a time series with periodic behavior. Analysis of the Wolf sunspot numbers with MARS appears to give an improvement over existing nonlinear Threshold and Bilinear models.

Download or read book Applied Predictive Modeling written by Max Kuhn and published by Springer Science & Business Media. This book was released on 2013-05-17 with total page 595 pages. Available in PDF, EPUB and Kindle. Book excerpt: Applied Predictive Modeling covers the overall predictive modeling process, beginning with the crucial steps of data preprocessing, data splitting and foundations of model tuning. The text then provides intuitive explanations of numerous common and modern regression and classification techniques, always with an emphasis on illustrating and solving real data problems. The text illustrates all parts of the modeling process through many hands-on, real-life examples, and every chapter contains extensive R code for each step of the process. This multi-purpose text can be used as an introduction to predictive models and the overall modeling process, a practitioner’s reference handbook, or as a text for advanced undergraduate or graduate level predictive modeling courses. To that end, each chapter contains problem sets to help solidify the covered concepts and uses data available in the book’s R package. This text is intended for a broad audience as both an introduction to predictive models as well as a guide to applying them. Non-mathematical readers will appreciate the intuitive explanations of the techniques while an emphasis on problem-solving with real data across a wide variety of applications will aid practitioners who wish to extend their expertise. Readers should have knowledge of basic statistical ideas, such as correlation and linear regression analysis. While the text is biased against complex equations, a mathematical background is needed for advanced topics.

Download or read book Mixed Effects Models for Complex Data written by Lang Wu and published by CRC Press. This book was released on 2009-11-11 with total page 431 pages. Available in PDF, EPUB and Kindle. Book excerpt: Although standard mixed effects models are useful in a range of studies, other approaches must often be used in correlation with them when studying complex or incomplete data. Mixed Effects Models for Complex Data discusses commonly used mixed effects models and presents appropriate approaches to address dropouts, missing data, measurement errors, censoring, and outliers. For each class of mixed effects model, the author reviews the corresponding class of regression model for cross-sectional data. An overview of general models and methods, along with motivating examples After presenting real data examples and outlining general approaches to the analysis of longitudinal/clustered data and incomplete data, the book introduces linear mixed effects (LME) models, generalized linear mixed models (GLMMs), nonlinear mixed effects (NLME) models, and semiparametric and nonparametric mixed effects models. It also includes general approaches for the analysis of complex data with missing values, measurement errors, censoring, and outliers. Self-contained coverage of specific topics Subsequent chapters delve more deeply into missing data problems, covariate measurement errors, and censored responses in mixed effects models. Focusing on incomplete data, the book also covers survival and frailty models, joint models of survival and longitudinal data, robust methods for mixed effects models, marginal generalized estimating equation (GEE) models for longitudinal or clustered data, and Bayesian methods for mixed effects models. Background material In the appendix, the author provides background information, such as likelihood theory, the Gibbs sampler, rejection and importance sampling methods, numerical integration methods, optimization methods, bootstrap, and matrix algebra. Failure to properly address missing data, measurement errors, and other issues in statistical analyses can lead to severely biased or misleading results. This book explores the biases that arise when naïve methods are used and shows which approaches should be used to achieve accurate results in longitudinal data analysis.

Download or read book Seamless R and C Integration with Rcpp written by Dirk Eddelbuettel and published by Springer Science & Business Media. This book was released on 2013-06-04 with total page 236 pages. Available in PDF, EPUB and Kindle. Book excerpt: Rcpp is the glue that binds the power and versatility of R with the speed and efficiency of C++. With Rcpp, the transfer of data between R and C++ is nearly seamless, and high-performance statistical computing is finally accessible to most R users. Rcpp should be part of every statistician's toolbox. -- Michael Braun, MIT Sloan School of Management "Seamless R and C++ integration with Rcpp" is simply a wonderful book. For anyone who uses C/C++ and R, it is an indispensable resource. The writing is outstanding. A huge bonus is the section on applications. This section covers the matrix packages Armadillo and Eigen and the GNU Scientific Library as well as RInside which enables you to use R inside C++. These applications are what most of us need to know to really do scientific programming with R and C++. I love this book. -- Robert McCulloch, University of Chicago Booth School of Business Rcpp is now considered an essential package for anybody doing serious computational research using R. Dirk's book is an excellent companion and takes the reader from a gentle introduction to more advanced applications via numerous examples and efficiency enhancing gems. The book is packed with all you might have ever wanted to know about Rcpp, its cousins (RcppArmadillo, RcppEigen .etc.), modules, package development and sugar. Overall, this book is a must-have on your shelf. -- Sanjog Misra, UCLA Anderson School of Management The Rcpp package represents a major leap forward for scientific computations with R. With very few lines of C++ code, one has R's data structures readily at hand for further computations in C++. Hence, high-level numerical programming can be made in C++ almost as easily as in R, but often with a substantial speed gain. Dirk is a crucial person in these developments, and his book takes the reader from the first fragile steps on to using the full Rcpp machinery. A very recommended book! -- Søren Højsgaard, Department of Mathematical Sciences, Aalborg University, Denmark "Seamless R and C ++ Integration with Rcpp" provides the first comprehensive introduction to Rcpp. Rcpp has become the most widely-used language extension for R, and is deployed by over one-hundred different CRAN and BioConductor packages. Rcpp permits users to pass scalars, vectors, matrices, list or entire R objects back and forth between R and C++ with ease. This brings the depth of the R analysis framework together with the power, speed, and efficiency of C++. Dirk Eddelbuettel has been a contributor to CRAN for over a decade and maintains around twenty packages. He is the Debian/Ubuntu maintainer for R and other quantitative software, edits the CRAN Task Views for Finance and High-Performance Computing, is a co-founder of the annual R/Finance conference, and an editor of the Journal of Statistical Software. He holds a Ph.D. in Mathematical Economics from EHESS (Paris), and works in Chicago as a Senior Quantitative Analyst.

Download or read book Modern Multivariate Statistical Techniques written by Alan J. Izenman and published by Springer Science & Business Media. This book was released on 2009-03-02 with total page 757 pages. Available in PDF, EPUB and Kindle. Book excerpt: This is the first book on multivariate analysis to look at large data sets which describes the state of the art in analyzing such data. Material such as database management systems is included that has never appeared in statistics books before.

Download or read book Principles of Data Mining written by David J. Hand and published by MIT Press. This book was released on 2001-08-17 with total page 594 pages. Available in PDF, EPUB and Kindle. Book excerpt: The first truly interdisciplinary text on data mining, blending the contributions of information science, computer science, and statistics. The growing interest in data mining is motivated by a common problem across disciplines: how does one store, access, model, and ultimately describe and understand very large data sets? Historically, different aspects of data mining have been addressed independently by different disciplines. This is the first truly interdisciplinary text on data mining, blending the contributions of information science, computer science, and statistics. The book consists of three sections. The first, foundations, provides a tutorial overview of the principles underlying data mining algorithms and their application. The presentation emphasizes intuition rather than rigor. The second section, data mining algorithms, shows how algorithms are constructed to solve specific problems in a principled manner. The algorithms covered include trees and rules for classification and regression, association rules, belief networks, classical statistical models, nonlinear models such as neural networks, and local "memory-based" models. The third section shows how all of the preceding analysis fits together when applied to real-world data mining problems. Topics include the role of metadata, how to handle missing data, and data preprocessing.

Download or read book Ecological Models and Data in R written by Benjamin M. Bolker and published by Princeton University Press. This book was released on 2008-07-21 with total page 408 pages. Available in PDF, EPUB and Kindle. Book excerpt: Introduction and background; Exploratory data analysis and graphics; Deterministic functions for ecological modeling; Probability and stochastic distributions for ecological modeling; Stochatsic simulation and power analysis; Likelihood and all that; Optimization and all that; Likelihood examples; Standar statistics revisited; Modeling variance; Dynamic models.

Download or read book Gaussian Processes for Machine Learning written by Carl Edward Rasmussen and published by MIT Press. This book was released on 2005-11-23 with total page 266 pages. Available in PDF, EPUB and Kindle. Book excerpt: A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.

Download or read book Statistical Pattern Recognition written by Andrew R. Webb and published by John Wiley & Sons. This book was released on 2003-07-25 with total page 516 pages. Available in PDF, EPUB and Kindle. Book excerpt: Statistical pattern recognition is a very active area of study andresearch, which has seen many advances in recent years. New andemerging applications - such as data mining, web searching,multimedia data retrieval, face recognition, and cursivehandwriting recognition - require robust and efficient patternrecognition techniques. Statistical decision making and estimationare regarded as fundamental to the study of pattern recognition. Statistical Pattern Recognition, Second Edition has been fullyupdated with new methods, applications and references. It providesa comprehensive introduction to this vibrant area - with materialdrawn from engineering, statistics, computer science and the socialsciences - and covers many application areas, such as databasedesign, artificial neural networks, and decision supportsystems. * Provides a self-contained introduction to statistical patternrecognition. * Each technique described is illustrated by real examples. * Covers Bayesian methods, neural networks, support vectormachines, and unsupervised classification. * Each section concludes with a description of the applicationsthat have been addressed and with further developments of thetheory. * Includes background material on dissimilarity, parameterestimation, data, linear algebra and probability. * Features a variety of exercises, from 'open-book' questions tomore lengthy projects. The book is aimed primarily at senior undergraduate and graduatestudents studying statistical pattern recognition, patternprocessing, neural networks, and data mining, in both statisticsand engineering departments. It is also an excellent source ofreference for technical professionals working in advancedinformation development environments. For further information on the techniques and applicationsdiscussed in this book please visit ahref="http://www.statistical-pattern-recognition.net/"www.statistical-pattern-recognition.net/a