Download or read book Minimax Estimation and Model Identification for High dimensional Regression written by Zhan Wang and published by . This book was released on 2012 with total page 160 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Direction Identification and Minimax Estimation by Generalized Eigenvalue Problem in High Dimensional Sparse Regression written by Mathieu Sauvenier and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Progress in Nonparametric Minimax Estimation and High Dimensional Hypothesis Testing written by Yandi Shen and published by . This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation is divided into two parts. In the first part, we study minimax estimation of functions and functionals in nonparametric regression models. The investigation of statistical limits in such models deepens theoretical understanding in related problems and leads to new probabilistic tools and methodologies of broader interest. In the second part, we study the asymptotics in some high dimensional testing problems involving the Gaussian distribution, such as the Gaussian sequence model with convex constraint and testing of covariance matrices. A general framework is developed to analyze the power behavior of test statistics via accurate non-asymptotic expansions.

Download or read book Multiple Testing and Minimax Estimation in Sparse Linear Regression written by Weijie Su and published by . This book was released on 2016 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In many real-world statistical problems, we observe a response variable of interest together with a large number of potentially explanatory variables of which a majority may be irrelevant. For this type of problem, controlling the false discovery rate (FDR) guarantees that most of the selected variables, often termed discoveries in a scientific context, are truly explanatory and thus replicable. Inspired by ideas from the Benjamini-Hochberg procedure (BHq), this thesis proposes a new method named SLOPE to control the FDR in sparse high-dimensional linear regression. SLOPE is a computationally efficient procedure that works by regularizing the fitted coefficients according to their ranks: the higher the rank, the larger the penalty. This adaptive regularization is analogous to the BHq, which compares more significant p-values with more stringent thresholds. Under orthogonal designs, SLOPE with the BHq critical values is proven to control FDR at any given level. Moreover, we demonstrate empirically that this method also appears to have appreciable inferential properties under more general design matrices while offering substantial power. The thesis proceeds to explore the estimation properties of SLOPE. Although SLOPE was developed from a multiple testing viewpoint, we show the surprising result that it achieves optimal squared errors under Gaussian random designs. This optimality holds under a weak assumption on the l0-sparsity level of the underlying signals, and is sharp in the sense that this is the best possible error any estimator can achieve. An appealing feature is that SLOPE does not require any knowledge of the degree of sparsity, and yet automatically adapts to yield optimal total squared errors over a wide range of l0-sparsity classes. Finally, we conclude this thesis by focusing on Nesterov's accelerated scheme, which is integral to a fast algorithmic implementation of SLOPE. Specifically, we prove that, as the step size vanishes, this scheme converges in a rigorous sense to a second-order ordinary differential equation (ODE). This continuous time ODE allows for a better understanding of Nesterov's scheme, and thus it can serve as a tool for analyzing and generalizing this scheme. A fruitful application of this tool yields a family of schemes with similar convergence rates. The ODE interpretation also suggests restarting Nesterov's scheme leading to a new algorithm, which is proven to converge at a linear rate whenever the objective is strongly convex.

Download or read book Statistical Inference for High Dimensional Problems written by Rajarshi Mukherjee and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: In this dissertation, we study minimax hypothesis testing in high-dimensional regression against sparse alternatives and minimax estimation of average treatment effect in an semiparametric regression with possibly large number of covariates.

Download or read book Methodology for Estimation and Model Selection in High dimensional Regression with Endogeneity written by Fan Du and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Since the advent of high-dimensional data structures in many areas such as medical and biological sciences, economics, and marketing investigation over the past few decades, the need for statistical modeling techniques of such data has grown. In high-dimensional statistical modeling techniques, model selection is an important aspect. The purpose of model selection is to select the most appropriate model from all possible high-dimensional statistical models where the number of explanatory variables is larger than the sample size. In high-dimensional model selection, endogeneity is a challenging issue. Endogeneity is defined as when a predictor variable (X) in a regression model is related to the model error term (Ïæ), which results in inconsistency of model selection. Because of the existence of endogeneity, Fan and Liao (2014) pointed out that exogenous assumptions in most statistical methods are not able to validate in high-dimensional model selection, and exogenous assumptions means a predictor variable (X) in a regression model is not related to the model error term (Ïæ). To avoid the effect of endogeneity, Fan and Liao (2014) proposed the focused generalized method-of-moments (FGMM) approach in high-dimensional linear models with endogeneity for selecting significant variables consistently. We propose the FGMM approach with modifications for high-dimensional linear and nonlinear models with endogeneity to choose all of the significant variables. The theorems in Fan and Liao (2014) show that FGMM approach consistently chooses the true model as the sample size goes to infinity in both the linear and nonlinear models. In linear models with endogeneity, we modify the penalty term to improve the selection performance. In nonlinear models with endogeneity, we adjust the loss function in the FGMM approach to achieve model selection consistency, which is to select the true model as the sample size n goes to infinity. This modified approach adopts instrumental variables to satisfy an exogenous assumption for consistently selecting the most appropriate model. The instrumental variables are defined as variable W that is correlated with the independent variable X and uncorrelated with the error term Ïæ. In other words, the instrument variables do not have endogenous problems. In the modified approach, instrumental variables are utilized to develop the loss function and penalized objective function for selecting consistent and significant variables in the model. Further, the modified approach can do model selection and estimation simultaneously. The simulations for high-dimensional linear and nonlinear models with endogeneity are conducted to illustrate the performance of the modified approach. In the simulations, we compare the performances of the modified FGMM approach and that of the penalized least square method with a variety of penalty functions, like Lasso, Adaptive Lasso, SCAD and MCP to select significant variables in the optimal model. The simulation results demonstrate that the modified FGMM approach has better performance in model selection and has higher estimation accuracy than those of the penalized least squared method in high-dimensional linear and nonlinear models. The simulation results also indicate that the utilization of different penalty terms, such as Adaptive Lasso, SCAD, and MCP, can improve estimation accuracy of parameters in the model compared with the Lasso. A real-world example is employed to evaluate the effectiveness of the modified FGMM approach.

Download or read book High dimensional Regression Models with Structured Coefficients written by Yuan Li and published by . This book was released on 2018 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Regression models are very common for statistical inference, especially linear regression models with Gaussian noise. But in many modern scientific applications with large-scale datasets, the number of samples is small relative to the number of model parameters, which is the so-called high- dimensional setting. Directly applying classical linear regression models to high-dimensional data is ill-posed. Thus it is necessary to impose additional assumptions for regression coefficients to make high-dimensional statistical analysis possible. Regularization methods with sparsity assumptions have received substantial attention over the past two decades. But there are still some open questions regarding high-dimensional statistical analysis. Firstly, most literature provides statistical analysis for high-dimensional linear models with Gaussian noise, it is unclear whether similar results still hold if we are no longer in the Gaussian setting. To answer this question under Poisson setting, we study the minimax rates and provide an implementable convex algorithm for high-dimensional Poisson inverse problems under weak sparsity assumption and physical constraints. Secondly, much of the theory and methodology for high-dimensional linear regression models are based on the assumption that independent variables are independent of each other or have weak correlations. But it is possible that this assumption is not satisfied that some features are highly correlated with each other. It is natural to ask whether it is still possible to make high-dimensional statistical inference with high-correlated designs. Thus we provide a graph-based regularization method for high-dimensional regression models with high-correlated designs along with theoretical guarantees.

Download or read book Introduction to High Dimensional Statistics written by Christophe Giraud and published by CRC Press. This book was released on 2021-08-25 with total page 410 pages. Available in PDF, EPUB and Kindle. Book excerpt: Praise for the first edition: "[This book] succeeds singularly at providing a structured introduction to this active field of research. ... it is arguably the most accessible overview yet published of the mathematical ideas and principles that one needs to master to enter the field of high-dimensional statistics. ... recommended to anyone interested in the main results of current research in high-dimensional statistics as well as anyone interested in acquiring the core mathematical skills to enter this area of research." —Journal of the American Statistical Association Introduction to High-Dimensional Statistics, Second Edition preserves the philosophy of the first edition: to be a concise guide for students and researchers discovering the area and interested in the mathematics involved. The main concepts and ideas are presented in simple settings, avoiding thereby unessential technicalities. High-dimensional statistics is a fast-evolving field, and much progress has been made on a large variety of topics, providing new insights and methods. Offering a succinct presentation of the mathematical foundations of high-dimensional statistics, this new edition: Offers revised chapters from the previous edition, with the inclusion of many additional materials on some important topics, including compress sensing, estimation with convex constraints, the slope estimator, simultaneously low-rank and row-sparse linear regression, or aggregation of a continuous set of estimators. Introduces three new chapters on iterative algorithms, clustering, and minimax lower bounds. Provides enhanced appendices, minimax lower-bounds mainly with the addition of the Davis-Kahan perturbation bound and of two simple versions of the Hanson-Wright concentration inequality. Covers cutting-edge statistical methods including model selection, sparsity and the Lasso, iterative hard thresholding, aggregation, support vector machines, and learning theory. Provides detailed exercises at the end of every chapter with collaborative solutions on a wiki site. Illustrates concepts with simple but clear practical examples.

Download or read book Minimax Estimation in Regression and Random Censorship Models written by Eduard N. Belitser and published by . This book was released on 2000 with total page 148 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Inverse Problems and High Dimensional Estimation written by Pierre Alquier and published by Springer Science & Business Media. This book was released on 2011-06-07 with total page 204 pages. Available in PDF, EPUB and Kindle. Book excerpt: The “Stats in the Château” summer school was held at the CRC château on the campus of HEC Paris, Jouy-en-Josas, France, from August 31 to September 4, 2009. This event was organized jointly by faculty members of three French academic institutions ─ ENSAE ParisTech, the Ecole Polytechnique ParisTech, and HEC Paris ─ which cooperate through a scientific foundation devoted to the decision sciences. The scientific content of the summer school was conveyed in two courses, one by Laurent Cavalier (Université Aix-Marseille I) on "Ill-posed Inverse Problems", and one by Victor Chernozhukov (Massachusetts Institute of Technology) on "High-dimensional Estimation with Applications to Economics". Ten invited researchers also presented either reviews of the state of the art in the field or of applications, or original research contributions. This volume contains the lecture notes of the two courses. Original research articles and a survey complement these lecture notes. Applications to economics are discussed in various contributions.

Download or read book A Note on Minimax estimation in Regression Models with Affine Restrictions written by Hilmar Drygas and published by . This book was released on 1988 with total page 72 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Minimax Estimation with Structured Data written by Jan-Christian Klaus Hütter and published by . This book was released on 2019 with total page 299 pages. Available in PDF, EPUB and Kindle. Book excerpt: Modern statistics often deals with high-dimensional problems that suffer from poor performance guarantees and from the curse of dimensionality. In this thesis, we study how structural assumptions can be used to overcome these difficulties in several estimation problems, spanning three different areas of statistics: shape-constrained estimation, causal discovery, and optimal transport. In the area of shape-constrained estimation, we study the estimation of matrices, first under the assumption of bounded total-variation (TV) and second under the assumption that the underlying matrix is Monge, or supermodular. While the first problem has a long history in image denoising, the latter structure has so far been mainly investigated in the context of computer science and optimization. For TV denoising, we provide fast rates that are adaptive to the underlying edge sparsity of the image, as well as generalizations to other graph structures, including higher-dimensional grid-graphs. For the estimation of Monge matrices, we give near minimax rates for their estimation, including the case where latent permutations act on the rows and columns of the matrix. In the latter case, we also give two computationally efficient and consistent estimators. Moreover, we show how to obtain estimation rates in the related problem of estimating continuous totally positive distributions in 2D. In the area of causal discovery, we investigate a linear cyclic causal model and give an estimator that is near minimax optimal for causal graphs of bounded in-degree. In the area of optimal transport, we introduce the notion of the transport rank of a coupling and provide empirical and theoretical evidence that it can be used to significantly improve rates of estimation of Wasserstein distances and optimal transport plans. Finally, we give near minimax optimal rates for the estimation of smooth optimal transport maps based on a wavelet regularization of the semi-dual objective.

Download or read book Minimax Ridge Regression Estimation written by George Casella and published by . This book was released on 1977 with total page 348 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Regression Graphics written by R. Dennis Cook and published by John Wiley & Sons. This book was released on 2009-09-25 with total page 378 pages. Available in PDF, EPUB and Kindle. Book excerpt: An exploration of regression graphics through computer graphics. Recent developments in computer technology have stimulated new and exciting uses for graphics in statistical analyses. Regression Graphics, one of the first graduate-level textbooks on the subject, demonstrates how statisticians, both theoretical and applied, can use these exciting innovations. After developing a relatively new regression context that requires few scope-limiting conditions, Regression Graphics guides readers through the process of analyzing regressions graphically and assessing and selecting models. This innovative reference makes use of a wide range of graphical tools, including 2D and 3D scatterplots, 3D binary response plots, and scatterplot matrices. Supplemented by a companion ftp site, it features numerous data sets and applied examples that are used to elucidate the theory. Other important features of this book include: * Extensive coverage of a relatively new regression context based on dimension-reduction subspaces and sufficient summary plots * Graphical regression, an iterative visualization process for constructing sufficient regression views * Graphics for regressions with a binary response * Graphics for model assessment, including residual plots * Net-effects plots for assessing predictor contributions * Graphics for predictor and response transformations * Inverse regression methods * Access to a Web site of supplemental plots, data sets, and 3D color displays. An ideal text for students in graduate-level courses on statistical analysis, Regression Graphics is also an excellent reference for professional statisticians.

Download or read book On minimax estimation in linear regression models with ellipsoidal constraints written by Norbert Christopeit and published by . This book was released on 1991 with total page 34 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Methods for Estimation and Inference for High dimensional Models written by Lina Lin and published by . This book was released on 2017 with total page 166 pages. Available in PDF, EPUB and Kindle. Book excerpt: This thesis tackles three different problems in high-dimensional statistics. The first two parts of the thesis focus on estimation of sparse high-dimensional undirected graphical models under non-standard conditions, specifically, non-Gaussianity and missingness, when observations are continuous. To address estimation under non-Gaussianity, we propose a general framework involving augmenting the score matching losses introduced in Hyva ̈rinen [2005, 2007] with an l1-regularizing penalty. This method, which we refer to as regularized score matching, allows for computationally efficient treatment of Gaussian and non-Gaussian continuous exponential family models because the considered loss becomes a penalized quadratic and thus yields piecewise linear solution paths. Under suitable irrepresentability conditions and distributional assumptions, we show that regularized score matching generates consistent graph estimates in sparse high-dimensional settings. Through numerical experiments and an application to RNAseq data, we confirm that regularized score matching achieves state-of- the-art performance in the Gaussian case and provides a valuable tool for computationally efficient estimation in non-Gaussian graphical models. To address estimation of sparse high-dimensional undirected graphical models with missing observations, we propose adapting the regularized score matching framework by substituting in surrogates of relevant statistics to accommodate these circumstances, as in Loh and Wainwright [2012] and Kolar and Xing [2012]. For Gaussian and non-Gaussian continuous exponential family models, the use of these surrogates may result in a loss of semi-definiteness, and thus nonconvexity, in the objective. Nevertheless, under suitable distributional assumptions, the global optimum is close to the truth in matrix l1 norm with high probability in sparse high-dimensional settings. Furthermore, under the same set of assumptions, we show that the composite gradient descent algorithm we propose for minimizing the modified objective converges at a geometric rate to a solution close to the global optimum with high probability. The last part of the thesis moves away from undirected graphical models, and is instead concerned with inference in high-dimensional regression models. Specifically, we investigate how to construct asymptotically valid confidence intervals and p-values for the fixed effects in a high-dimensional linear mixed effect model. The framework we propose, largely founded on a recent work [Bu ̈hlmann, 2013], entails de-biasing a ‘naive’ ridge estimator. We show via numerical experiments that the method controls for Type I error in hypothesis testing and generates confidence intervals that achieve target coverage, outperforming competitors that assume observations are homogeneous when observations are, in fact, correlated within group.

Download or read book High Dimensional Statistics written by Martin J. Wainwright and published by Cambridge University Press. This book was released on 2019-02-21 with total page 571 pages. Available in PDF, EPUB and Kindle. Book excerpt: Recent years have witnessed an explosion in the volume and variety of data collected in all scientific disciplines and industrial settings. Such massive data sets present a number of challenges to researchers in statistics and machine learning. This book provides a self-contained introduction to the area of high-dimensional statistics, aimed at the first-year graduate level. It includes chapters that are focused on core methodology and theory - including tail bounds, concentration inequalities, uniform laws and empirical process, and random matrices - as well as chapters devoted to in-depth exploration of particular model classes - including sparse linear models, matrix models with rank constraints, graphical models, and various types of non-parametric models. With hundreds of worked examples and exercises, this text is intended both for courses and for self-study by graduate students and researchers in statistics, machine learning, and related fields who must understand, apply, and adapt modern statistical methods suited to large-scale data.