Download or read book Semiparametric Approaches for Dimension Reduction Through Gradient Descent on Manifold written by Qing Xiao and published by . This book was released on 2021 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: High-dimensional data arises at an unprecedented speed across various fields. Statistical models might fail on high-dimensional data due to the "curse of dimensionality". Sufficient dimension reduction (SDR) is to extract the core information through low-dimensional mapping so that efficient statistical models can be built while preserving the regression information in the high-dimensional data. We develop several SDR methods through manifold parameterization. First, we propose a SDR method, gemDR, based on local kernel regression without loss of information of the conditional mean E[Y|X]. The method, gemDR, focuses on identifying the central mean subspace (CMS). Then gemDR is extended to CS-gemDR for central subspace (CS), through the empirical cumulative distribution function. CS-OPG, a modified outer product gradient (OPG) method for CS, is developed as an initial estimator for CS-gemDR. The basis B of the CMS or CS is estimated by a gradient descent algorithm. An update scheme on a Grassmann manifold is to preserve the orthogonality constraint on the parameters. To determine the dimension of the CMS and CS, two consistent cross-validation criteria are developed. Our methods show better performance for highly correlated features. We also develop ER-OPG and ER-MAVE to identify the basis of CS on a manifold. The entire conditional distribution of a response given predictors is estimated in a heterogeneous regression setting through composite expectile regression. The computation algorithm is developed through an orthogonal updating scheme on a manifold. The proposed methods are adaptive to the structure of the random errors and do not require restrictive probabilistic assumptions as inverse methods. Our methods are first-order methods which are computationally efficient compared with second-order methods. Their efficacy is demonstrated through numerical simulation and real data applications. The kernel bandwidth and basis are estimated simultaneously. The proposed methods show better performance in estimation of the basis and its dimension.
Download or read book Elements of Dimensionality Reduction and Manifold Learning written by Benyamin Ghojogh and published by Springer Nature. This book was released on 2023-02-02 with total page 617 pages. Available in PDF, EPUB and Kindle. Book excerpt: Dimensionality reduction, also known as manifold learning, is an area of machine learning used for extracting informative features from data for better representation of data or separation between classes. This book presents a cohesive review of linear and nonlinear dimensionality reduction and manifold learning. Three main aspects of dimensionality reduction are covered: spectral dimensionality reduction, probabilistic dimensionality reduction, and neural network-based dimensionality reduction, which have geometric, probabilistic, and information-theoretic points of view to dimensionality reduction, respectively. The necessary background and preliminaries on linear algebra, optimization, and kernels are also explained to ensure a comprehensive understanding of the algorithms. The tools introduced in this book can be applied to various applications involving feature extraction, image processing, computer vision, and signal processing. This book is applicable to a wide audience who would like to acquire a deep understanding of the various ways to extract, transform, and understand the structure of data. The intended audiences are academics, students, and industry professionals. Academic researchers and students can use this book as a textbook for machine learning and dimensionality reduction. Data scientists, machine learning scientists, computer vision scientists, and computer scientists can use this book as a reference. It can also be helpful to statisticians in the field of statistical learning and applied mathematicians in the fields of manifolds and subspace analysis. Industry professionals, including applied engineers, data engineers, and engineers in various fields of science dealing with machine learning, can use this as a guidebook for feature extraction from their data, as the raw data in industry often require preprocessing. The book is grounded in theory but provides thorough explanations and diverse examples to improve the reader’s comprehension of the advanced topics. Advanced methods are explained in a step-by-step manner so that readers of all levels can follow the reasoning and come to a deep understanding of the concepts. This book does not assume advanced theoretical background in machine learning and provides necessary background, although an undergraduate-level background in linear algebra and calculus is recommended.
Download or read book Nonlinear Dimensionality Reduction written by John A. Lee and published by Springer Science & Business Media. This book was released on 2007-10-31 with total page 316 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book describes established and advanced methods for reducing the dimensionality of numerical databases. Each description starts from intuitive ideas, develops the necessary mathematical details, and ends by outlining the algorithmic implementation. The text provides a lucid summary of facts and concepts relating to well-known methods as well as recent developments in nonlinear dimensionality reduction. Methods are all described from a unifying point of view, which helps to highlight their respective strengths and shortcomings. The presentation will appeal to statisticians, computer scientists and data analysts, and other practitioners having a basic background in statistics or computational learning.
Download or read book Computational Methods for Manifold Learning written by Xin Yang and published by . This book was released on 2007 with total page 132 pages. Available in PDF, EPUB and Kindle. Book excerpt: In many real world applications, data samples lying in a high dimensional ambient space can be modeled by very low dimensional nonlinear manifolds. Manifold learning, as a new framework of machine learning, discovers this low dimensional structure from the collection of the high dimensional data. In this thesis, some novel manifold learning methods are proposed, including conical dimension, semi-supervised nonlinear dimensionality reduction, active learning for the semi-supervised manifold learning, and mesh-free manifold learning. {it Conical dimension} is proposed as a novel local intrinsic dimension estimator, for estimating the intrinsic dimension of a data set consisting of points lying in the proximity of a manifold. It can also be applied to intersection and boundary detection. The accuracy and robustness of the algorithm are illustrated by both synthetic and real-world data experiments. Both synthetic and real life examples are shown. We propose the {it semi-supervised nonlinear dimensionality reduction} by introducing the prior information into basic nonlinear dimensionality reduction method, such as LLE and LTSA. The sensitivity analysis of our algorithms shows that prior information will improve the stability of the solution. We demonstrate the usefulness of our algorithm by synthetic and real life examples. A principled approach for selecting the data points for labeling used in semi-supervised manifold learning is proposed as {it active learning} method. Experiments on both synthetic and real-world problems show that our proposed methods can substantially improve the accuracy of the computed global parameterizations over several alternative methods. In the last section, we propose an alternative dimensionality reduction method, namely mesh-free manifold learning, which introduce the phase field models into dimensionality reduction problem to track the data movement during the time step of the dimensionality reduction procedure.
Download or read book Dimensionality Reduction with Unsupervised Nearest Neighbors written by Oliver Kramer and published by Springer Science & Business Media. This book was released on 2013-05-30 with total page 137 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book is devoted to a novel approach for dimensionality reduction based on the famous nearest neighbor method that is a powerful classification and regression approach. It starts with an introduction to machine learning concepts and a real-world application from the energy domain. Then, unsupervised nearest neighbors (UNN) is introduced as efficient iterative method for dimensionality reduction. Various UNN models are developed step by step, reaching from a simple iterative strategy for discrete latent spaces to a stochastic kernel-based algorithm for learning submanifolds with independent parameterizations. Extensions that allow the embedding of incomplete and noisy patterns are introduced. Various optimization approaches are compared, from evolutionary to swarm-based heuristics. Experimental comparisons to related methodologies taking into account artificial test data sets and also real-world data demonstrate the behavior of UNN in practical scenarios. The book contains numerous color figures to illustrate the introduced concepts and to highlight the experimental results.
Download or read book Semiparametric Approaches to Dimension Reduction written by and published by . This book was released on 1992 with total page 21 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Nonlinear Dimensionality Reduction by Manifold Unfolding written by Pooyan Khajehpour Tadavani and published by . This book was released on 2013 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Every second, an enormous volume of data is being gathered from various sources and stored in huge data banks. Most of the time, monitoring a data source requires several parallel measurements, which form a high-dimensional sample vector. Due to the curse of dimensionality, applying machine learning methods, that is, studying and analyzing high-dimensional data, could be difficult. The essential task of dimensionality reduction is to faithfully represent a given set of high-dimensional data samples with a few variables. The goal of this thesis is to develop and propose new techniques for handling high-dimensional data, in order to address contemporary demand in machine learning applications. Most prominent nonlinear dimensionality reduction methods do not explicitly provide a way to handle out-of-samples. The starting point of this thesis is a nonlinear technique, called Embedding by Affine Transformations (EAT), which reduces the dimensionality of out-of-sample data as well. In this method, a convex optimization is solved for estimating a transformation between the high-dimensional input space and the low-dimensional embedding space. To the best of our knowledge, EAT is the only distance-preserving method for nonlinear dimensionality reduction capable of handling out-of-samples. The second method that we propose is TesseraMap. This method is a scalable extension of EAT. Conceptually, TesseraMap partitions the underlying manifold of data into a set of tesserae and then unfolds it by constructing a tessellation in a low-dimensional subspace of the embedding space. Crucially, the desired tessellation is obtained through solving a small semidefinite program; therefore, this method can efficiently handle tens of thousands of data points in a short time. The final outcome of this thesis is a novel method in dimensionality reduction called Isometric Patch Alignment (IPA). Intuitively speaking, IPA first considers a number of overlapping flat patches, which cover the underlying manifold of the high-dimensional input data. Then, IPA rearranges the patches and stitches the neighbors together on their overlapping parts. We prove that stitching two neighboring patches aligns them together; thereby, IPA unfolds the underlying manifold of data. Although this method and TesseraMap have similar approaches, IPA is more scalable; it embeds one million data points in only a few minutes. More importantly, unlike EAT and TesseraMap, which unfold the underlying manifold by stretching it, IPA constructs the unfolded manifold through patch alignment. We show this novel approach is advantageous in many cases. In addition, compared to the other well-known dimensionality reduction methods, IPA has several important characteristics; for example, it is noise tolerant, it handles non-uniform samples, and it can embed non-convex manifolds properly. In addition to these three dimensionality reduction methods, we propose a method for subspace clustering called Low-dimensional Localized Clustering (LDLC). In subspace clustering, data is partitioned into clusters, such that the points of each cluster lie close to a low-dimensional subspace. The unique property of LDLC is that it produces localized clusters on the underlying manifold of data. By conducting several experiments, we show this property is an asset in many machine learning tasks. This method can also be used for local dimensionality reduction. Moreover, LDLC is a suitable tool for forming the tesserae in TesseraMap, and also for creating the patches in IPA.
Download or read book Independent Component Analysis written by Aapo Hyvärinen and published by John Wiley & Sons. This book was released on 2004-04-05 with total page 505 pages. Available in PDF, EPUB and Kindle. Book excerpt: A comprehensive introduction to ICA for students and practitioners Independent Component Analysis (ICA) is one of the most exciting new topics in fields such as neural networks, advanced statistics, and signal processing. This is the first book to provide a comprehensive introduction to this new technique complete with the fundamental mathematical background needed to understand and utilize it. It offers a general overview of the basics of ICA, important solutions and algorithms, and in-depth coverage of new applications in image processing, telecommunications, audio signal processing, and more. Independent Component Analysis is divided into four sections that cover: * General mathematical concepts utilized in the book * The basic ICA model and its solution * Various extensions of the basic ICA model * Real-world applications for ICA models Authors Hyvarinen, Karhunen, and Oja are well known for their contributions to the development of ICA and here cover all the relevant theory, new algorithms, and applications in various fields. Researchers, students, and practitioners from a variety of disciplines will find this accessible volume both helpful and informative.
Download or read book Introduction to Machine Learning written by Ethem Alpaydin and published by MIT Press. This book was released on 2014-08-22 with total page 639 pages. Available in PDF, EPUB and Kindle. Book excerpt: Introduction -- Supervised learning -- Bayesian decision theory -- Parametric methods -- Multivariate methods -- Dimensionality reduction -- Clustering -- Nonparametric methods -- Decision trees -- Linear discrimination -- Multilayer perceptrons -- Local models -- Kernel machines -- Graphical models -- Brief contents -- Hidden markov models -- Bayesian estimation -- Combining multiple learners -- Reinforcement learning -- Design and analysis of machine learning experiments.
Download or read book Neural Networks for Pattern Recognition written by Christopher M. Bishop and published by Oxford University Press. This book was released on 1995-11-23 with total page 501 pages. Available in PDF, EPUB and Kindle. Book excerpt: Statistical pattern recognition; Probability density estimation; Single-layer networks; The multi-layer perceptron; Radial basis functions; Error functions; Parameter optimization algorithms; Pre-processing and feature extraction; Learning and generalization; Bayesian techniques; Appendix; References; Index.
Download or read book Automated Machine Learning written by Frank Hutter and published by Springer. This book was released on 2019-05-17 with total page 223 pages. Available in PDF, EPUB and Kindle. Book excerpt: This open access book presents the first comprehensive overview of general methods in Automated Machine Learning (AutoML), collects descriptions of existing systems based on these methods, and discusses the first series of international challenges of AutoML systems. The recent success of commercial ML applications and the rapid growth of the field has created a high demand for off-the-shelf ML methods that can be used easily and without expert knowledge. However, many of the recent machine learning successes crucially rely on human experts, who manually select appropriate ML architectures (deep learning architectures or more traditional ML workflows) and their hyperparameters. To overcome this problem, the field of AutoML targets a progressive automation of machine learning, based on principles from optimization and machine learning itself. This book serves as a point of entry into this quickly-developing field for researchers and advanced students alike, as well as providing a reference for practitioners aiming to use AutoML in their work.
Download or read book Stochastic Approximation written by Vivek S. Borkar and published by Springer. This book was released on 2009-01-01 with total page 177 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Large scale Kernel Machines written by Léon Bottou and published by MIT Press. This book was released on 2007 with total page 409 pages. Available in PDF, EPUB and Kindle. Book excerpt: Solutions for learning from large scale datasets, including kernel learning algorithms that scale linearly with the volume of the data and experiments carried out on realistically large datasets. Pervasive and networked computers have dramatically reduced the cost of collecting and distributing large datasets. In this context, machine learning algorithms that scale poorly could simply become irrelevant. We need learning algorithms that scale linearly with the volume of the data while maintaining enough statistical efficiency to outperform algorithms that simply process a random subset of the data. This volume offers researchers and engineers practical solutions for learning from large scale datasets, with detailed descriptions of algorithms and experiments carried out on realistically large datasets. At the same time it offers researchers information that can address the relative lack of theoretical grounding for many useful algorithms. After a detailed description of state-of-the-art support vector machine technology, an introduction of the essential concepts discussed in the volume, and a comparison of primal and dual optimization techniques, the book progresses from well-understood techniques to more novel and controversial approaches. Many contributors have made their code and data available online for further experimentation. Topics covered include fast implementations of known algorithms, approximations that are amenable to theoretical guarantees, and algorithms that perform well in practice but are difficult to analyze theoretically. Contributors Léon Bottou, Yoshua Bengio, Stéphane Canu, Eric Cosatto, Olivier Chapelle, Ronan Collobert, Dennis DeCoste, Ramani Duraiswami, Igor Durdanovic, Hans-Peter Graf, Arthur Gretton, Patrick Haffner, Stefanie Jegelka, Stephan Kanthak, S. Sathiya Keerthi, Yann LeCun, Chih-Jen Lin, Gaëlle Loosli, Joaquin Quiñonero-Candela, Carl Edward Rasmussen, Gunnar Rätsch, Vikas Chandrakant Raykar, Konrad Rieck, Vikas Sindhwani, Fabian Sinz, Sören Sonnenburg, Jason Weston, Christopher K. I. Williams, Elad Yom-Tov
Download or read book Static Analysis written by Agostino Cortesi and published by Springer Science & Business Media. This book was released on 1999-09-08 with total page 366 pages. Available in PDF, EPUB and Kindle. Book excerpt: Static analysis is increasingly recognized as a fundamental reasearch area aimed at studying and developing tools for high performance implementations and v- i cation systems for all programming language paradigms. The last two decades have witnessed substantial developments in this eld, ranging from theoretical frameworks to design, implementation, and application of analyzers in optim- ing compilers. Since 1994, SAS has been the annual conference and forum for researchers in all aspects of static analysis. This volume contains the proceedings of the 6th International Symposium on Static Analysis (SAS’99) which was held in Venice, Italy, on 22{24 September 1999. The previous SAS conferences were held in Namur (Belgium), Glasgow (UK), Aachen (Germany), Paris (France), and Pisa (Italy). The program committee selected 18 papers out of 42 submissions on the basis of at least three reviews. The resulting volume o ers to the reader a complete landscape of the research in this area. The papers contribute to the following topics: foundations of static analysis, abstract domain design, and applications of static analysis to di erent programming paradigms (concurrent, synchronous, imperative, object oriented, logical, and functional). In particular, several papers use static analysis for obtaining state space reduction in concurrent systems. New application elds are also addressed, such as the problems of security and secrecy.
Download or read book Regression Graphics written by R. Dennis Cook and published by John Wiley & Sons. This book was released on 1998-09-30 with total page 380 pages. Available in PDF, EPUB and Kindle. Book excerpt: Zur graphischen Präsentation von Regressionsdaten gibt es seit dem Vormarsch der Computertechnik vielfältige neue Möglichkeiten, die über die klassischen Ansätze hinausgehen. Der Autor betritt mit seinen Ideen häufig Neuland; er illustriert sie mit zahlreichen Beispielen, Diagrammen und Abbildungen (die entsprechenden 3D- und Farbversionen sind über Internet abrufbar). (11/98)
Download or read book Gaussian Processes for Machine Learning written by Carl Edward Rasmussen and published by MIT Press. This book was released on 2005-11-23 with total page 266 pages. Available in PDF, EPUB and Kindle. Book excerpt: A comprehensive and self-contained introduction to Gaussian processes, which provide a principled, practical, probabilistic approach to learning in kernel machines. Gaussian processes (GPs) provide a principled, practical, probabilistic approach to learning in kernel machines. GPs have received increased attention in the machine-learning community over the past decade, and this book provides a long-needed systematic and unified treatment of theoretical and practical aspects of GPs in machine learning. The treatment is comprehensive and self-contained, targeted at researchers and students in machine learning and applied statistics. The book deals with the supervised-learning problem for both regression and classification, and includes detailed algorithms. A wide variety of covariance (kernel) functions are presented and their properties discussed. Model selection is discussed both from a Bayesian and a classical perspective. Many connections to other well-known techniques from machine learning and statistics are discussed, including support-vector machines, neural networks, splines, regularization networks, relevance vector machines and others. Theoretical issues including learning curves and the PAC-Bayesian framework are treated, and several approximation methods for learning with large datasets are discussed. The book contains illustrative examples and exercises, and code and datasets are available on the Web. Appendixes provide mathematical background and a discussion of Gaussian Markov processes.
Download or read book Seamless R and C Integration with Rcpp written by Dirk Eddelbuettel and published by Springer Science & Business Media. This book was released on 2013-06-04 with total page 236 pages. Available in PDF, EPUB and Kindle. Book excerpt: Rcpp is the glue that binds the power and versatility of R with the speed and efficiency of C++. With Rcpp, the transfer of data between R and C++ is nearly seamless, and high-performance statistical computing is finally accessible to most R users. Rcpp should be part of every statistician's toolbox. -- Michael Braun, MIT Sloan School of Management "Seamless R and C++ integration with Rcpp" is simply a wonderful book. For anyone who uses C/C++ and R, it is an indispensable resource. The writing is outstanding. A huge bonus is the section on applications. This section covers the matrix packages Armadillo and Eigen and the GNU Scientific Library as well as RInside which enables you to use R inside C++. These applications are what most of us need to know to really do scientific programming with R and C++. I love this book. -- Robert McCulloch, University of Chicago Booth School of Business Rcpp is now considered an essential package for anybody doing serious computational research using R. Dirk's book is an excellent companion and takes the reader from a gentle introduction to more advanced applications via numerous examples and efficiency enhancing gems. The book is packed with all you might have ever wanted to know about Rcpp, its cousins (RcppArmadillo, RcppEigen .etc.), modules, package development and sugar. Overall, this book is a must-have on your shelf. -- Sanjog Misra, UCLA Anderson School of Management The Rcpp package represents a major leap forward for scientific computations with R. With very few lines of C++ code, one has R's data structures readily at hand for further computations in C++. Hence, high-level numerical programming can be made in C++ almost as easily as in R, but often with a substantial speed gain. Dirk is a crucial person in these developments, and his book takes the reader from the first fragile steps on to using the full Rcpp machinery. A very recommended book! -- Søren Højsgaard, Department of Mathematical Sciences, Aalborg University, Denmark "Seamless R and C ++ Integration with Rcpp" provides the first comprehensive introduction to Rcpp. Rcpp has become the most widely-used language extension for R, and is deployed by over one-hundred different CRAN and BioConductor packages. Rcpp permits users to pass scalars, vectors, matrices, list or entire R objects back and forth between R and C++ with ease. This brings the depth of the R analysis framework together with the power, speed, and efficiency of C++. Dirk Eddelbuettel has been a contributor to CRAN for over a decade and maintains around twenty packages. He is the Debian/Ubuntu maintainer for R and other quantitative software, edits the CRAN Task Views for Finance and High-Performance Computing, is a co-founder of the annual R/Finance conference, and an editor of the Journal of Statistical Software. He holds a Ph.D. in Mathematical Economics from EHESS (Paris), and works in Chicago as a Senior Quantitative Analyst.
