Download or read book Minimum Divergence Methods in Statistical Machine Learning written by Shinto Eguchi and published by Springer Nature. This book was released on 2022-03-14 with total page 224 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book explores minimum divergence methods of statistical machine learning for estimation, regression, prediction, and so forth, in which we engage in information geometry to elucidate their intrinsic properties of the corresponding loss functions, learning algorithms, and statistical models. One of the most elementary examples is Gauss's least squares estimator in a linear regression model, in which the estimator is given by minimization of the sum of squares between a response vector and a vector of the linear subspace hulled by explanatory vectors. This is extended to Fisher's maximum likelihood estimator (MLE) for an exponential model, in which the estimator is provided by minimization of the Kullback-Leibler (KL) divergence between a data distribution and a parametric distribution of the exponential model in an empirical analogue. Thus, we envisage a geometric interpretation of such minimization procedures such that a right triangle is kept with Pythagorean identity in the sense of the KL divergence. This understanding sublimates a dualistic interplay between a statistical estimation and model, which requires dual geodesic paths, called m-geodesic and e-geodesic paths, in a framework of information geometry. We extend such a dualistic structure of the MLE and exponential model to that of the minimum divergence estimator and the maximum entropy model, which is applied to robust statistics, maximum entropy, density estimation, principal component analysis, independent component analysis, regression analysis, manifold learning, boosting algorithm, clustering, dynamic treatment regimes, and so forth. We consider a variety of information divergence measures typically including KL divergence to express departure from one probability distribution to another. An information divergence is decomposed into the cross-entropy and the (diagonal) entropy in which the entropy associates with a generative model as a family of maximum entropy distributions; the cross entropy associates with a statistical estimation method via minimization of the empirical analogue based on given data. Thus any statistical divergence includes an intrinsic object between the generative model and the estimation method. Typically, KL divergence leads to the exponential model and the maximum likelihood estimation. It is shown that any information divergence leads to a Riemannian metric and a pair of the linear connections in the framework of information geometry. We focus on a class of information divergence generated by an increasing and convex function U, called U-divergence. It is shown that any generator function U generates the U-entropy and U-divergence, in which there is a dualistic structure between the U-divergence method and the maximum U-entropy model. We observe that a specific choice of U leads to a robust statistical procedure via the minimum U-divergence method. If U is selected as an exponential function, then the corresponding U-entropy and U-divergence are reduced to the Boltzmann-Shanon entropy and the KL divergence; the minimum U-divergence estimator is equivalent to the MLE. For robust supervised learning to predict a class label we observe that the U-boosting algorithm performs well for contamination of mislabel examples if U is appropriately selected. We present such maximal U-entropy and minimum U-divergence methods, in particular, selecting a power function as U to provide flexible performance in statistical machine learning.
Download or read book Information Theory and Statistical Learning written by Frank Emmert-Streib and published by Springer Science & Business Media. This book was released on 2009 with total page 443 pages. Available in PDF, EPUB and Kindle. Book excerpt: This interdisciplinary text offers theoretical and practical results of information theoretic methods used in statistical learning. It presents a comprehensive overview of the many different methods that have been developed in numerous contexts.
Download or read book Minimum Gamma Divergence for Regression and Classification Problems written by Shinto Eguchi and published by Springer. This book was released on 2024-11-29 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book introduces the gamma-divergence, a measure of distance between probability distributions that was proposed by Fujisawa and Eguchi in 2008. The gamma-divergence has been extensively explored to provide robust estimation when the power index γ is positive. The gamma-divergence can be defined even when the power index γ is negative, as long as the condition of integrability is satisfied. Thus, the authors consider the gamma-divergence defined on a set of discrete distributions. The arithmetic, geometric, and harmonic means for the distribution ratios are closely connected with the gamma-divergence with a negative γ. In particular, the authors call the geometric-mean (GM) divergence the gamma-divergence when γ is equal to -1. The book begins by providing an overview of the gamma-divergence and its properties. It then goes on to discuss the applications of the gamma-divergence in various areas, including machine learning, statistics, and ecology. Bernoulli, categorical, Poisson, negative binomial, and Boltzmann distributions are discussed as typical examples. Furthermore, regression analysis models that explicitly or implicitly assume these distributions as the dependent variable in generalized linear models are discussed to apply the minimum gamma-divergence method. In ensemble learning, AdaBoost is derived by the exponential loss function in the weighted majority vote manner. It is pointed out that the exponential loss function is deeply connected to the GM divergence. In the Boltzmann machine, the maximum likelihood has to use approximation methods such as mean field approximation because of the intractable computation of the partition function. However, by considering the GM divergence and the exponential loss, it is shown that the calculation of the partition function is not necessary, and it can be executed without variational inference.
Download or read book Introduction to Statistical Machine Learning written by Masashi Sugiyama and published by Morgan Kaufmann. This book was released on 2015-10-31 with total page 535 pages. Available in PDF, EPUB and Kindle. Book excerpt: Machine learning allows computers to learn and discern patterns without actually being programmed. When Statistical techniques and machine learning are combined together they are a powerful tool for analysing various kinds of data in many computer science/engineering areas including, image processing, speech processing, natural language processing, robot control, as well as in fundamental sciences such as biology, medicine, astronomy, physics, and materials. Introduction to Statistical Machine Learning provides a general introduction to machine learning that covers a wide range of topics concisely and will help you bridge the gap between theory and practice. Part I discusses the fundamental concepts of statistics and probability that are used in describing machine learning algorithms. Part II and Part III explain the two major approaches of machine learning techniques; generative methods and discriminative methods. While Part III provides an in-depth look at advanced topics that play essential roles in making machine learning algorithms more useful in practice. The accompanying MATLAB/Octave programs provide you with the necessary practical skills needed to accomplish a wide range of data analysis tasks. - Provides the necessary background material to understand machine learning such as statistics, probability, linear algebra, and calculus - Complete coverage of the generative approach to statistical pattern recognition and the discriminative approach to statistical machine learning - Includes MATLAB/Octave programs so that readers can test the algorithms numerically and acquire both mathematical and practical skills in a wide range of data analysis tasks - Discusses a wide range of applications in machine learning and statistics and provides examples drawn from image processing, speech processing, natural language processing, robot control, as well as biology, medicine, astronomy, physics, and materials
Download or read book Geometric Science of Information written by Frank Nielsen and published by Springer Nature. This book was released on 2023-07-31 with total page 641 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book constitutes the proceedings of the 6th International Conference on Geometric Science of Information, GSI 2023, held in St. Malo, France, during August 30-September 1, 2023. The 125 full papers presented in this volume were carefully reviewed and selected from 161 submissions. They cover all the main topics and highlights in the domain of geometric science of information, including information geometry manifolds of structured data/information and their advanced applications. The papers are organized in the following topics: geometry and machine learning; divergences and computational information geometry; statistics, topology and shape spaces; geometry and mechanics; geometry, learning dynamics and thermodynamics; quantum information geometry; geometry and biological structures; geometry and applications.
Download or read book Statistical Inference written by Ayanendranath Basu and published by CRC Press. This book was released on 2011-06-22 with total page 424 pages. Available in PDF, EPUB and Kindle. Book excerpt: In many ways, estimation by an appropriate minimum distance method is one of the most natural ideas in statistics. However, there are many different ways of constructing an appropriate distance between the data and the model: the scope of study referred to by "Minimum Distance Estimation" is literally huge. Filling a statistical resource gap, Stati
Download or read book Algorithmic Learning Theory written by Ricard Gavalda and published by Springer Science & Business Media. This book was released on 2003-10-07 with total page 325 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book constitutes the refereed proceedings of the 14th International Conference on Algorithmic Learning Theory, ALT 2003, held in Sapporo, Japan in October 2003. The 19 revised full papers presented together with 2 invited papers and abstracts of 3 invited talks were carefully reviewed and selected from 37 submissions. The papers are organized in topical sections on inductive inference, learning and information extraction, learning with queries, learning with non-linear optimization, learning from random examples, and online prediction.
Download or read book Rank Based Methods for Shrinkage and Selection written by A. K. Ehsanes Saleh and published by John Wiley & Sons Incorporated. This book was released on 2022 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: "The purpose of this book is to lay the groundwork for robust data science using rankbased methods. The field of machine learning has not yet fully embraced a class of robust estimators that would address issues that limit the value of least-squares estimation. For example, outliers in data sets may produce misleading results that are not suitable for inference. They can also affect results obtained from penalty estimators. We believe that robust estimators for regression problems are well-suited to data science. This book is intended to provide both practical and mathematical foundations in the study of rank-based methods. It will introduce a number of new ideas and approaches to the practice and theory of robust estimation and encourage readers to pursue further investigation in this field. While the main goal of this book is to provide a rigorous treatment of the subject matter, we begin with some introductory material to build insight and intuition about rank-based regression and penalty estimators, especially for those who are new to the topic and those looking to understand key concepts. To motivate the need for such methods, we will start with a discussion of the median as it is the key to rank-based methods and then build on that concept towards the notion of robust data science"--
Download or read book Data Analysis and Related Applications 4 written by Yiannis Dimotikalis and published by John Wiley & Sons. This book was released on 2024-10-08 with total page 420 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Pattern Recognition and Machine Learning written by Christopher M. Bishop and published by Springer. This book was released on 2016-08-23 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This is the first textbook on pattern recognition to present the Bayesian viewpoint. The book presents approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible. It uses graphical models to describe probability distributions when no other books apply graphical models to machine learning. No previous knowledge of pattern recognition or machine learning concepts is assumed. Familiarity with multivariate calculus and basic linear algebra is required, and some experience in the use of probabilities would be helpful though not essential as the book includes a self-contained introduction to basic probability theory.
Download or read book Machine Learning for Signal Processing written by Max A. Little and published by Oxford University Press. This book was released on 2019-08-13 with total page 378 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book describes in detail the fundamental mathematics and algorithms of machine learning (an example of artificial intelligence) and signal processing, two of the most important and exciting technologies in the modern information economy. Taking a gradual approach, it builds up concepts in a solid, step-by-step fashion so that the ideas and algorithms can be implemented in practical software applications. Digital signal processing (DSP) is one of the 'foundational' engineering topics of the modern world, without which technologies such the mobile phone, television, CD and MP3 players, WiFi and radar, would not be possible. A relative newcomer by comparison, statistical machine learning is the theoretical backbone of exciting technologies such as automatic techniques for car registration plate recognition, speech recognition, stock market prediction, defect detection on assembly lines, robot guidance, and autonomous car navigation. Statistical machine learning exploits the analogy between intelligent information processing in biological brains and sophisticated statistical modelling and inference. DSP and statistical machine learning are of such wide importance to the knowledge economy that both have undergone rapid changes and seen radical improvements in scope and applicability. Both make use of key topics in applied mathematics such as probability and statistics, algebra, calculus, graphs and networks. Intimate formal links between the two subjects exist and because of this many overlaps exist between the two subjects that can be exploited to produce new DSP tools of surprising utility, highly suited to the contemporary world of pervasive digital sensors and high-powered, yet cheap, computing hardware. This book gives a solid mathematical foundation to, and details the key concepts and algorithms in this important topic.
Download or read book The Minimum Description Length Principle written by Peter D. Grünwald and published by MIT Press. This book was released on 2007 with total page 736 pages. Available in PDF, EPUB and Kindle. Book excerpt: This introduction to the MDL Principle provides a reference accessible to graduate students and researchers in statistics, pattern classification, machine learning, and data mining, to philosophers interested in the foundations of statistics, and to researchers in other applied sciences that involve model selection.
Download or read book Learning Machine Translation written by Cyril Goutte and published by MIT Press. This book was released on 2009 with total page 329 pages. Available in PDF, EPUB and Kindle. Book excerpt: How Machine Learning can improve machine translation: enabling technologies and new statistical techniques.
Download or read book Density Ratio Estimation in Machine Learning written by Masashi Sugiyama and published by Cambridge University Press. This book was released on 2012-02-20 with total page 343 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book introduces theories, methods and applications of density ratio estimation, a newly emerging paradigm in the machine learning community.
Download or read book Information Geometry and Its Applications written by Nihat Ay and published by Springer. This book was released on 2018-11-03 with total page 450 pages. Available in PDF, EPUB and Kindle. Book excerpt: The book gathers contributions from the fourth conference on Information Geometry and its Applications, which was held on June 12–17, 2016, at Liblice Castle, Czech Republic on the occasion of Shun-ichi Amari’s 80th birthday and was organized by the Czech Academy of Sciences’ Institute of Information Theory and Automation. The conference received valuable financial support from the Max Planck Institute for Mathematics in the Sciences (Information Theory of Cognitive Systems Group), Czech Academy of Sciences’ Institute of Information Theory and Automation, and Università degli Studi di Roma Tor Vergata. The aim of the conference was to highlight recent advances in the field of information geometry and to identify new research directions. To this end, the event brought together leading experts in the field who, in invited talks and poster sessions, discussed both theoretical work and achievements in the many fields of application in which information geometry plays an essential role.
Download or read book Machine Learning and Knowledge Discovery in Databases Research Track written by Nuria Oliver and published by Springer Nature. This book was released on 2021-09-09 with total page 838 pages. Available in PDF, EPUB and Kindle. Book excerpt: The multi-volume set LNAI 12975 until 12979 constitutes the refereed proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2021, which was held during September 13-17, 2021. The conference was originally planned to take place in Bilbao, Spain, but changed to an online event due to the COVID-19 pandemic. The 210 full papers presented in these proceedings were carefully reviewed and selected from a total of 869 submissions. The volumes are organized in topical sections as follows: Research Track: Part I: Online learning; reinforcement learning; time series, streams, and sequence models; transfer and multi-task learning; semi-supervised and few-shot learning; learning algorithms and applications. Part II: Generative models; algorithms and learning theory; graphs and networks; interpretation, explainability, transparency, safety. Part III: Generative models; search and optimization; supervised learning; text mining and natural language processing; image processing, computer vision and visual analytics. Applied Data Science Track: Part IV: Anomaly detection and malware; spatio-temporal data; e-commerce and finance; healthcare and medical applications (including Covid); mobility and transportation. Part V: Automating machine learning, optimization, and feature engineering; machine learning based simulations and knowledge discovery; recommender systems and behavior modeling; natural language processing; remote sensing, image and video processing; social media.
Download or read book Linguistically Motivated Statistical Machine Translation written by Deyi Xiong and published by Springer. This book was released on 2015-02-11 with total page 159 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book provides a wide variety of algorithms and models to integrate linguistic knowledge into Statistical Machine Translation (SMT). It helps advance conventional SMT to linguistically motivated SMT by enhancing the following three essential components: translation, reordering and bracketing models. It also serves the purpose of promoting the in-depth study of the impacts of linguistic knowledge on machine translation. Finally it provides a systematic introduction of Bracketing Transduction Grammar (BTG) based SMT, one of the state-of-the-art SMT formalisms, as well as a case study of linguistically motivated SMT on a BTG-based platform.
