Download or read book High dimensional Regression Models with Structured Coefficients written by Yuan Li and published by . This book was released on 2018 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Regression models are very common for statistical inference, especially linear regression models with Gaussian noise. But in many modern scientific applications with large-scale datasets, the number of samples is small relative to the number of model parameters, which is the so-called high- dimensional setting. Directly applying classical linear regression models to high-dimensional data is ill-posed. Thus it is necessary to impose additional assumptions for regression coefficients to make high-dimensional statistical analysis possible. Regularization methods with sparsity assumptions have received substantial attention over the past two decades. But there are still some open questions regarding high-dimensional statistical analysis. Firstly, most literature provides statistical analysis for high-dimensional linear models with Gaussian noise, it is unclear whether similar results still hold if we are no longer in the Gaussian setting. To answer this question under Poisson setting, we study the minimax rates and provide an implementable convex algorithm for high-dimensional Poisson inverse problems under weak sparsity assumption and physical constraints. Secondly, much of the theory and methodology for high-dimensional linear regression models are based on the assumption that independent variables are independent of each other or have weak correlations. But it is possible that this assumption is not satisfied that some features are highly correlated with each other. It is natural to ask whether it is still possible to make high-dimensional statistical inference with high-correlated designs. Thus we provide a graph-based regularization method for high-dimensional regression models with high-correlated designs along with theoretical guarantees.
Download or read book Tests of Hypotheses on Regression Coefficients in High Dimensional Regression Models written by Ye Alex Zhao and published by . This book was released on 2022 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Statistical inference in high-dimensional settings has become an important area of research due to the increased production of high-dimensional data in a wide variety of areas. However, few approaches towards simultaneous hypothesis testing of high-dimensional regression coefficients have been proposed. In the first project of this dissertation, we introduce a new method for simultaneous tests of the coefficients in a high-dimensional linear regression model. Our new test statistic is based on the sum-of-squares of the score function mean with an additional power-enhancement term. The asymptotic distribution and power of the test statistic are derived, and our procedure is shown to outperform existing approaches. We conduct Monte Carlo simulations to demonstrate performance improvements over existing methods and apply the testing procedure to a real data example. In the second project, we propose a test statistic for regression coefficients in a high-dimensional setting that applies for generalized linear models. Building on previous work on testing procedures for high-dimensional linear regression models, we extend this approach to create a new testing methodology for GLMs, with specific illustrations for the Poisson and logistic regression scenarios. The asymptotic distribution of the test statistic is established, and both simulation results and a real data analysis are conducted to illustrate the performance of our proposed method. The final project of this dissertation introduces two new approaches for testing high-dimensional regression coefficients in the partial linear model setting and more generally for linear hypothesis tests in linear models. Our proposed statistic is motivated by the profile least squares method and decorrelation score method for high-dimensional inference, which we show to be equivalent in these particular cases. We outline the empirical performance of the new test statistic with simulation studies and real data examples. These results indicate generally satisfactory performance under a wide range of settings and applicability to real world data problems.
Download or read book Robust Penalized Regression for Complex High dimensional Data written by Bin Luo and published by . This book was released on 2020 with total page 169 pages. Available in PDF, EPUB and Kindle. Book excerpt: "Robust high-dimensional data analysis has become an important and challenging task in complex Big Data analysis due to the high-dimensionality and data contamination. One of the most popular procedures is the robust penalized regression. In this dissertation, we address three typical robust ultra-high dimensional regression problems via penalized regression approaches. The first problem is related to the linear model with the existence of outliers, dealing with the outlier detection, variable selection and parameter estimation simultaneously. The second problem is related to robust high-dimensional mean regression with irregular settings such as the data contamination, data asymmetry and heteroscedasticity. The third problem is related to robust bi-level variable selection for the linear regression model with grouping structures in covariates. In Chapter 1, we introduce the background and challenges by overviews of penalized least squares methods and robust regression techniques. In Chapter 2, we propose a novel approach in a penalized weighted least squares framework to perform simultaneous variable selection and outlier detection. We provide a unified link between the proposed framework and a robust M-estimation in general settings. We also establish the non-asymptotic oracle inequalities for the joint estimation of both the regression coefficients and weight vectors. In Chapter 3, we establish a framework of robust estimators in high-dimensional regression models using Penalized Robust Approximated quadratic M estimation (PRAM). This framework allows general settings such as random errors lack of symmetry and homogeneity, or covariates are not sub-Gaussian. Theoretically, we show that, in the ultra-high dimension setting, the PRAM estimator has local estimation consistency at the minimax rate enjoyed by the LS-Lasso and owns the local oracle property, under certain mild conditions. In Chapter 4, we extend the study in Chapter 3 to robust high-dimensional data analysis with structured sparsity. In particular, we propose a framework of high-dimensional M-estimators for bi-level variable selection. This framework encourages bi-level sparsity through a computationally efficient two-stage procedure. It produces strong robust parameter estimators if some nonconvex redescending loss functions are applied. In theory, we provide sufficient conditions under which our proposed two-stage penalized M-estimator possesses simultaneous local estimation consistency and the bi-level variable selection consistency, if a certain nonconvex penalty function is used at the group level. The performances of the proposed estimators are demonstrated in both simulation studies and real examples. In Chapter 5, we provide some discussions and future work."--Abstract from author supplied metadata
Download or read book Inference on Structural Changes in High Dimensional Linear Regression Models written by Hongjin Zhang and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation is dedicated to studying the problem of constructing asymptotically valid confidence intervals for change points in high-dimensional linear models, where the number of parameters may vastly exceed the sampling period.In Chapter 2, we develop an algorithmic estimator for a single change point and establish the optimal rate of estimation, Op(Îl 8́22 ), where Îl represents the jump size under a high dimensional scaling. The optimal result ensures the existence of limiting distributions. Asymptotic distributions are derived under both vanishing and non-vanishing regimes of jump size. In the former case, it corresponds to the argmax of a two-sided Brownian motion, while in the latter case to the argmax of a two-sided random walk, both with negative drifts. We also provide the relationship between the two distributions, which allows construction of regime (vanishing vs non-vanishing) adaptive confidence intervals.In Chapter 3, we extend our analysis to the statistical inference for multiple change points in high-dimensional linear regression models. We develop locally refitted estimators and evaluate their convergence rates both component-wise and simultaneously. Following similar manner as in Chapter 2, we achieve an optimal rate of estimation under the component-wise scenario, which guarantees the existence of limiting distributions. While we also establish the simultaneous rate which is the sharpest available by a logarithmic factor. Component-wise and joint limiting distributions are derived under vanishing and non-vanishing regimes of jump sizes, demonstrating the relationship between distributions in the two regimes.Lastly in Chapter 4, we introduce a novel implementation method for finding preliminary change points estimates via integer linear programming, which has not yet been explored in the current literature.Overall, this dissertation provides a comprehensive framework for inference on single and multiple change points in high-dimensional linear models, offering novel and efficient algorithms with strong theoretical guarantees. All theoretical results are supported by Monte Carlo simulations.
Download or read book Sparse Graphical Modeling for High Dimensional Data written by Faming Liang and published by CRC Press. This book was released on 2023-08-02 with total page 151 pages. Available in PDF, EPUB and Kindle. Book excerpt: A general framework for learning sparse graphical models with conditional independence tests Complete treatments for different types of data, Gaussian, Poisson, multinomial, and mixed data Unified treatments for data integration, network comparison, and covariate adjustment Unified treatments for missing data and heterogeneous data Efficient methods for joint estimation of multiple graphical models Effective methods of high-dimensional variable selection Effective methods of high-dimensional inference
Download or read book Statistical Methods for Complex And or High Dimensional Data written by Shanshan Qin and published by . This book was released on 2020 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation focuses on the development and implementation of statistical methods for high-dimensional and/or complex data, with an emphasis on $p$, the number of explanatory variables, larger than $n$, the number of observations, the ratio of $p/n$ tending to a finite number, and data with outlier observations. First, we propose a non-negative feature selection and/or feature grouping (nnFSG) method. It deals with a general series of sign-constrained high-dimensional regression problems, which allows the regression coefficients to carry a structure of disjoint homogeneity, including sparsity as a special case. To solve the resulting non-convex optimization problem, we provide an algorithm that incorporates the difference of convex programming, augmented Lagrange and coordinate descent methods. Furthermore, we show that the aforementioned nnFSG method recovers the oracle estimate consistently, and yields a bound on the mean squared errors (MSE).} Besides, we examine the performance of our method by using finite sample simulations and a real protein mass spectrum dataset. Next, we consider a High-dimensional multivariate ridge regression model under the regime where both $p$ and $n$ are large enough with $p/n \rightarrow \kappa (0
Download or read book Component Identification and Estimation in Nonlinear High dimensional Regression Models by Structural Adaptation written by Alexander Samarov and published by . This book was released on 2003 with total page 39 pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book High dimensional Regression Modeling written by Patrick Breheny and published by . This book was released on 2018 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:
Download or read book Testing a Single Regression Coefficient in High Dimensional Regression Model written by Wei Lan and published by . This book was released on 2016 with total page 46 pages. Available in PDF, EPUB and Kindle. Book excerpt: In linear regression models with high dimensional data, the classical z-test (or t-test) for testing the significance of each single regression coefficient is no longer applicable. This is mainly because the number of covariates exceeds the sample size. In this paper, we propose a simple and novel alternative by introducing the Correlated Predictors Screening (CPS) method to control for predictors that are highly correlated with the target covariate. Accordingly, the classical ordinary least squares approach can be employed to estimate the regression coefficient associated with the target covariate. In addition, we demonstrate that the resulting estimator is consistent and asymptotically normal even if the random errors are heteroscedastic. This enables us to apply the z-test to assess the significance of each covariate. Based on the p-value obtained from testing the significance of each covariate, we further conduct multiple hypothesis testing by controlling the false discovery rate at the nominal level. Then, we show that the multiple hypothesis testing achieves consistent model selection. Simulation studies and empirical examples are presented to illustrate the finite sample performance and the usefulness of the proposed method, respectively.
Download or read book Large Dimensional Factor Analysis written by Jushan Bai and published by Now Publishers Inc. This book was released on 2008 with total page 90 pages. Available in PDF, EPUB and Kindle. Book excerpt: Large Dimensional Factor Analysis provides a survey of the main theoretical results for large dimensional factor models, emphasizing results that have implications for empirical work. The authors focus on the development of the static factor models and on the use of estimated factors in subsequent estimation and inference. Large Dimensional Factor Analysis discusses how to determine the number of factors, how to conduct inference when estimated factors are used in regressions, how to assess the adequacy pf observed variables as proxies for latent factors, how to exploit the estimated factors to test unit root tests and common trends, and how to estimate panel cointegration models.
Download or read book Inference for High dimensional Sparse Econometric Models written by Alexandre Belloni and published by . This book was released on 2011 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This article is about estimation and inference methods for high dimensional sparse (HDS) regression models in econometrics. High dimensional sparse models arise in situations where many regressors (or series terms) are available and the regression function is well-approximated by a parsimonious, yet unknown set of regressors. The latter condition makes it possible to estimate the entire regression function effectively by searching for approximately the right set of regressors. We discuss methods for identifying this set of regressors and estimating their coefficients based on l1 -penalization and describe key theoretical results. In order to capture realistic practical situations, we expressly allow for imperfect selection of regressors and study the impact of this imperfect selection on estimation and inference results. We focus the main part of the article on the use of HDS models and methods in the instrumental variables model and the partially linear model. We present a set of novel inference results for these models and illustrate their use with applications to returns to schooling and growth regression. -- inference under imperfect model selection ; structural effects ; high-dimensional econometrics ; instrumental regression ; partially linear regression ; returns-to-schooling ; growth regression
Download or read book Multivariate Reduced Rank Regression written by Gregory C. Reinsel and published by Springer Nature. This book was released on 2022-11-30 with total page 420 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book provides an account of multivariate reduced-rank regression, a tool of multivariate analysis that enjoys a broad array of applications. In addition to a historical review of the topic, its connection to other widely used statistical methods, such as multivariate analysis of variance (MANOVA), discriminant analysis, principal components, canonical correlation analysis, and errors-in-variables models, is also discussed. This new edition incorporates Big Data methodology and its applications, as well as high-dimensional reduced-rank regression, generalized reduced-rank regression with complex data, and sparse and low-rank regression methods. Each chapter contains developments of basic theoretical results, as well as details on computational procedures, illustrated with numerical examples drawn from disciplines such as biochemistry, genetics, marketing, and finance. This book is designed for advanced students, practitioners, and researchers, who may deal with moderate and high-dimensional multivariate data. Because regression is one of the most popular statistical methods, the multivariate regression analysis tools described should provide a natural way of looking at large (both cross-sectional and chronological) data sets. This book can be assigned in seminar-type courses taken by advanced graduate students in statistics, machine learning, econometrics, business, and engineering.
Download or read book Some Inference Problems in High Dimensional Linear Models written by Miles Edward Lopes and published by . This book was released on 2015 with total page 124 pages. Available in PDF, EPUB and Kindle. Book excerpt: During the past two decades, technological advances have led to a proliferation of high-dimensional problems in data analysis. The characteristic feature of such problems is that they involve large numbers of unknown parameters and relatively few observations. As the study of high-dimensional statistical models has developed, linear models have taken on a special status for their widespread application and extensive theory. Even so, much of the theoretical research on high-dimensional linear models has been concentrated on the problems of prediction and estimation, and many inferential questions regarding hypothesis tests and confidence intervals remain open. In this dissertation, we explore two sets of inferential questions arising in high-dimensional linear models. The first set deals with the residual bootstrap (RB) method and the distributional approximation of regression contrasts. The second set addresses the issue of unknown sparsity in the signal processing framework of compressed sensing. Although these topics involve distinct methods and applications, the dissertation is unified by an overall focus on the interplay between model structure and inference. Specifically, our work is motivated by an interest in using inferential methods to confirm the existence of model structure, and in developing new inferential methods that have minimal reliance on structural assumptions. The residual bootstrap method is a general approach to approximating the sampling distribution of statistics derived from estimated regression coefficients. When the number of regression coefficients p is small compared to the number of observations n, classical results show that RB consistently approximates the laws of contrasts obtained from least-squares coefficients. However, when p/n~1, it is known that there exist contrasts for which RB fails -- when applied to least-squares residuals. As a remedy, we propose an alternative method that is tailored to regression models involving near low-rank design matrices. In this situation, we prove that resampling the residuals of a ridge regression estimator can alleviate some of the problems that occur for least-squares residuals. Notably, our approach does not depend on sparsity in the true regression coefficients. Furthermore, the assumption of a near low-rank design is one that is satisfied in many applications and can be inspected directly in practice. In the second portion of the dissertation, we turn our attention to the subject of compressed sensing, which deals with the recovery of sparse high-dimensional signals from a limited number of linear measurements. Although the theory of compressed sensing offers strong recovery guarantees, many of its basic results depend on prior knowledge of the signal's sparsity level -- a parameter that is rarely known in practice. Towards a resolution of this issue, we introduce a generalized family of sparsity parameters that can be estimated in a way that is free of structural assumptions. We show that our estimator is ratio-consistent with a dimension-free rate of convergence, and also derive the estimator's limiting distribution. In turn, these results make it possible to set confidence intervals for the sparsity level and to test the hypothesis of sparsity in a precise sense.
Download or read book Large dimensional Panel Data Econometrics Testing Estimation And Structural Changes written by Feng Qu and published by World Scientific. This book was released on 2020-08-24 with total page 167 pages. Available in PDF, EPUB and Kindle. Book excerpt: This book aims to fill the gap between panel data econometrics textbooks, and the latest development on 'big data', especially large-dimensional panel data econometrics. It introduces important research questions in large panels, including testing for cross-sectional dependence, estimation of factor-augmented panel data models, structural breaks in panels and group patterns in panels. To tackle these high dimensional issues, some techniques used in Machine Learning approaches are also illustrated. Moreover, the Monte Carlo experiments, and empirical examples are also utilised to show how to implement these new inference methods. Large-Dimensional Panel Data Econometrics: Testing, Estimation and Structural Changes also introduces new research questions and results in recent literature in this field.
Download or read book Statistics for High Dimensional Data written by Peter Bühlmann and published by Springer Science & Business Media. This book was released on 2011-06-08 with total page 568 pages. Available in PDF, EPUB and Kindle. Book excerpt: Modern statistics deals with large and complex data sets, and consequently with models containing a large number of parameters. This book presents a detailed account of recently developed approaches, including the Lasso and versions of it for various models, boosting methods, undirected graphical modeling, and procedures controlling false positive selections. A special characteristic of the book is that it contains comprehensive mathematical theory on high-dimensional statistics combined with methodology, algorithms and illustrations with real data examples. This in-depth approach highlights the methods’ great potential and practical applicability in a variety of settings. As such, it is a valuable resource for researchers, graduate students and experts in statistics, applied mathematics and computer science.
Download or read book Partially Linear Models written by Wolfgang Härdle and published by Springer Science & Business Media. This book was released on 2012-12-06 with total page 210 pages. Available in PDF, EPUB and Kindle. Book excerpt: In the last ten years, there has been increasing interest and activity in the general area of partially linear regression smoothing in statistics. Many methods and techniques have been proposed and studied. This monograph hopes to bring an up-to-date presentation of the state of the art of partially linear regression techniques. The emphasis is on methodologies rather than on the theory, with a particular focus on applications of partially linear regression techniques to various statistical problems. These problems include least squares regression, asymptotically efficient estimation, bootstrap resampling, censored data analysis, linear measurement error models, nonlinear measurement models, nonlinear and nonparametric time series models.
Download or read book Regression Models with a Universal Penalized Function and Applications in Economics and Finance written by Mingwei Sun and published by . This book was released on 2018 with total page 152 pages. Available in PDF, EPUB and Kindle. Book excerpt: Variable selection is an important topic in linear regression analysis and attracts considerable research in this era of big data. It is fundamental to high-dimensional statistical modeling, including nonparametric regression. Some classic techniques include stepwise deletion and subset selection. These procedures, however, ignore stochastic errors inherited in the stages of variable selections, and the resulting subset suffers from lack of stability and low prediction accuracy. Penalized least squares provide new approaches to the variable selection problems with high-dimensional data. The least absolute shrinkage and selection operator (LASSO), which imposes an L$_1$-penalty on the regression coefficients, and the Elastic Net which combines an L$_1$ and an L$_2$ penalties are popular members of the penalized regressions. In this dissertation, we develop penalized linear regression with a universal penalty function, which includes the widely used ridge and Lasso penalty functions as special cases. A Monte Carlo simulation approach is developed to illustrate that the Elastic Net is also a special case of our model. The structure and properties of the universal penalty are studied, and the corresponding algorithm to solve the regression coefficients is developed. Furthermore, we apply our model to a real U.S. economic and financial data example. Simulation studies and real-data support the advantageous performance of the proposed method.
