Download or read book Performance of the Propensity Score Methods Using Random Forest and Logistic Regression Approaches on the Treatment Effect Estimation in Observational Study written by and published by . This book was released on 2017 with total page 35 pages. Available in PDF, EPUB and Kindle. Book excerpt: The propensity score (PS) is the probability of a subject receiving the treatment given the baseline covariates. People with the same propensity score tend to have the same distribution of covariates. Thus, propensity score related methods can be used to eliminate the systematic difference between treatment and control group so that improving the causal inferences in the observational study. In this project, a series of simulation studies are conducted to evaluate two widely used propensity score methods, matching and inverse probability of treatment weighting (IPTW), on their relative ability to estimate the treatment effect from non-randomized trials. One observes that the random forest based propensity score weighting can yield more promising treatment effect estimates compared with other PS methods. Besides that, simulated samples are also implemented to compare the performance of several matching methods on the balancing the covariates. It turns out that logistic regression based propensity score matching can reduce most of systematic differences between treatment and control group although it is not the top performer in the causal effect estimation. Finally, we illustrate the application of the propensity score methods discussed in the paper with an empirical example.

Download or read book Ensemble Learning for Propensity Score Methods in Observational Studies written by and published by . This book was released on 2018 with total page 103 pages. Available in PDF, EPUB and Kindle. Book excerpt: Propensity score methods have shown to reduce selection bias in observational studies. However, the consistency of the propensity score (PS) estimators strongly depends on a correct specification of the PS model. Logistic regression (LR) and recently machine learning tools are commonly used to estimate the propensity scores. We introduce a stacked generalization ensemble learning approach to improve propensity score estimation by fitting a meta learner on the predictions of a suitable set of diverse base learners. We perform a comprehensive Monte Carlo simulation study, implementing eight scenarios that mimic characteristics of typical data sets in educational studies. The treatment effect is estimated using the PS in Inverse Probability of Treatment Weighting (IPTW) with ATE weights. Performance of the models was evaluated by PS prediction accuracy, percent absolute bias, mean squared error and standard errors of treatment effect estimates, weight distribution and achieved covariate balance. Our proposed ensembles, especially using LR and GBM as meta learners trained on a set of 13 base learner predictions, led to superior reduction of bias compared to all underlying base learners. We examine modifications of the underlying base learner set and support recent literature that both, superior PS prediction accuracy and superior balance do not necessarily lead to more precise treatment effect estimates. Our findings suggest that stacked ensembles will allow educational researchers to obtain more precise treatment effect estimates in propensity score studies. We apply our best models to assess the average treatment effect of a Supplemental Instruction (SI) program in an introductory psychology (PSY 101) course at San Diego State University. We show that our methods balance the data after weighting and then confirm results in the recent literature that SI has a significantly positive impact on student success in the PSY101 course.

Download or read book The Performance of Propensity Score Methods to Estimate the Average Treatment Effect in Observational Studies with Selection Bias written by Sungur Gurel and published by . This book was released on 2012 with total page 61 pages. Available in PDF, EPUB and Kindle. Book excerpt: We investigated the performance of four different propensity score (PS) methods to reduce selection bias in estimates of the average treatment effect (ATE) in observational studies: inverse probability of treatment weighting (IPTW), truncated inverse probability of treatment weighting (TIPTW), optimal full propensity score matching (OFPSM), and propensity score stratification (PSS). We compared these methods in combination with three methods of standard error estimation: weighted least squares regression (WLS), Taylor series linearization (TSL), and jackknife (JK). We conducted a Monte Carlo Simulation study manipulating the number of subjects and the ratio of treated to total sample size. The results indicated that IPTW and OFPSM methods removed almost all of the bias while TIPTW and PSS removed about 90% of the bias. Some of TSL and JK standard errors were acceptable, some marginally overestimated, and some moderately overestimated. For the lower ratio of treated on sample sizes, all of the WLS standard errors were strongly underestimated, as designs get balanced, the underestimation gets less serious. Especially for the OFPSM, all of the TSL and JK standard errors were overestimated and WLS standard errors under estimated under all simulated conditions.

Download or read book Estimating Average Treatment Effects With Propensity Scores Estimated With Four Machine Learning Procedures written by Kip Brown and published by . This book was released on 2018 with total page 24 pages. Available in PDF, EPUB and Kindle. Book excerpt: Background: The increased availability of claims data allows one to build high dimensional datasets, rich in covariates, for accurately estimating treatment effects in medical and epidemiological cohort studies. This paper shows the full potential of machine learning for the estimation of average treatment effects with propensity score methods in a context rich and high dimensional datasets. Methods: Four different methods are used to estimate average treatment effects in the context of time to event outcomes. The four methods explored in this study are LASSO, Random Forest, Gradient Descent Boosting and Artificial Neural networks. Simulations based on an actual medical claims data set are used to assess the efficiency of these methods. The simulations are performed with over 100, 000 observations and 1,100 explanatory variables. Each method is tested on 500 datasets that are created from the original dataset, allowing us to report the mean and standard deviation of estimated average treatment effects. Results: The results are very promising for all four methods; however, LASSO, Random Forest and Gradient Boosting seem to be performing better than Random Forest. Conclusion: Machine Learning methods can be helpful for observational studies that use the propensity score when a very large number of covariates are available, the total number of observations is large, and the dependent event rare. This is an important result given the availability of big data related to Health Economics and Outcomes Research (HEOR) around the world.

Download or read book Comparison of Approaches for Handling Missingness in Covariates for Propensity Score Models written by Jiangxiu Zhou and published by . This book was released on 2015 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Causal effect estimation with observational data is subject to bias due to confounding. Although potential confounders could be adjusted for by fitting a multiple regression model, a more effective way to control for confounding is to use propensity score methods. Propensity scores are most commonly estimated from logistic regression with a binary exposure; generalized propensity scores could be estimated instead using linear regression if the exposure is continuous. One unresolved issue in propensity score estimation is handling of missing values in covariates. As covariates are only used for propensity score estimation but not for later outcome analysis, missing values in covariates may need to be handled differently from missing values in outcome analysis. Several approaches have been proposed for handling covariate missingness, including multiple imputation (MI), multiple imputation with missingness pattern (MIMP) and treatment mean imputation. There are other potentially useful approaches that have not been evaluated, including single imputation, single conditional mean imputation and Generalized Boosted Modeling (GBM), which is a nonparametric approach of estimating propensity scores and missing values are automatically accounted for in the estimation.To evaluate the performance of single imputation, single conditional mean imputation and GBM in comparison to the previously proposed approaches including treatment mean imputation, MI and MIMP, a simulation study is conducted with a binary exposure. Results suggest that when all confounders are included for propensity score estimation, single imputation, single conditional mean imputation, MI and MIMP perform almost equally well and better than treatment mean imputation and GBM. To examine whether the finding could be extended to a continuous exposure setting, another simulation study is conducted. Results suggest that single imputation, single conditional imputation, MI, MIMP and GBM with single conditional mean imputation have equally good and better performance than treatment mean imputation and GBM with incomplete data under scenario A (linearity and additivity). None of the approaches perform well under scenario G (nonlinearity and nonadditivity). These approaches are further demonstrated and compared through an empirical analysis with the Adolescent Alcohol Prevention Trial (AAPT). A similar pattern of results is observed as in the simulation study. It is recommended to impute missing covariates using different approaches and similar estimates help provide more confidence in the estimates.

Download or read book Developing a Protocol for Observational Comparative Effectiveness Research A User s Guide written by Agency for Health Care Research and Quality (U.S.) and published by Government Printing Office. This book was released on 2013-02-21 with total page 236 pages. Available in PDF, EPUB and Kindle. Book excerpt: This User’s Guide is a resource for investigators and stakeholders who develop and review observational comparative effectiveness research protocols. It explains how to (1) identify key considerations and best practices for research design; (2) build a protocol based on these standards and best practices; and (3) judge the adequacy and completeness of a protocol. Eleven chapters cover all aspects of research design, including: developing study objectives, defining and refining study questions, addressing the heterogeneity of treatment effect, characterizing exposure, selecting a comparator, defining and measuring outcomes, and identifying optimal data sources. Checklists of guidance and key considerations for protocols are provided at the end of each chapter. The User’s Guide was created by researchers affiliated with AHRQ’s Effective Health Care Program, particularly those who participated in AHRQ’s DEcIDE (Developing Evidence to Inform Decisions About Effectiveness) program. Chapters were subject to multiple internal and external independent reviews. More more information, please consult the Agency website: www.effectivehealthcare.ahrq.gov)

Download or read book Practical Propensity Score Methods Using R written by Walter Leite and published by SAGE Publications. This book was released on 2016-10-28 with total page 225 pages. Available in PDF, EPUB and Kindle. Book excerpt: Practical Propensity Score Methods Using R by Walter Leite is a practical book that uses a step-by-step analysis of realistic examples to help students understand the theory and code for implementing propensity score analysis with the R statistical language. With a comparison of both well-established and cutting-edge propensity score methods, the text highlights where solid guidelines exist to support best practices and where there is scarcity of research. Readers will find that this scaffolded approach to R and the book’s free online resources help them apply the text’s concepts to the analysis of their own data.

Download or read book Propensity Score Methods in Observational Studies written by Susanne Stampf and published by . This book was released on 2010 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Propensity Score Estimation with Random Forests written by Hei Ning Cham and published by . This book was released on 2013 with total page 96 pages. Available in PDF, EPUB and Kindle. Book excerpt: Random Forests is a statistical learning method which has been proposed for propensity score estimation models that involve complex interactions, nonlinear relationships, or both of the covariates. In this dissertation I conducted a simulation study to examine the effects of three Random Forests model specifications in propensity score analysis. The results suggested that, depending on the nature of data, optimal specification of (1) decision rules to select the covariate and its split value in a Classification Tree, (2) the number of covariates randomly sampled for selection, and (3) methods of estimating Random Forests propensity scores could potentially produce an unbiased average treatment effect estimate after propensity scores weighting by the odds adjustment. Compared to the logistic regression estimation model using the true propensity score model, Random Forests had an additional advantage in producing unbiased estimated standard error and correct statistical inference of the average treatment effect. The relationship between the balance on the covariates' means and the bias of average treatment effect estimate was examined both within and between conditions of the simulation. Within conditions, across repeated samples there was no noticeable correlation between the covariates' mean differences and the magnitude of bias of average treatment effect estimate for the covariates that were imbalanced before adjustment. Between conditions, small mean differences of covariates after propensity score adjustment were not sensitive enough to identify the optimal Random Forests model specification for propensity score analysis.

Download or read book Bias and Variance of Treatment Effect Estimators Using Propensity score Matching written by Diqiong Xie and published by . This book was released on 2011 with total page 257 pages. Available in PDF, EPUB and Kindle. Book excerpt: Observational studies are an indispensable complement to randomized clinical trials (RCT) for comparison of treatment effectiveness. Often RCTs cannot be carried out due to the costs of the trial, ethical questions and rarity of the outcome. When noncompliance and missing data are prevalent, RCTs become more like observational studies. The main problem is to adjust for the selection bias in the observational study. One increasingly used method is propensity-score matching. Compared to traditional multi-covariate matching methods, matching on the propensity score alleviates the curse of dimensionality. It allows investigators to balance multiple covariate distributions between treatment groups by matching on a single score. This thesis focuses on the large sample properties of the matching estimators of the treatment effect. The first part of this thesis deals with problems of the analytic supports of the logit propensity score and various matching methods. The second part of this thesis focuses on the matching estimators of additive and multiplicative treatment effects. We derive the asymptotic order of the biases and asymptotic distributions of the matching estimators. We also derive the large sample variance estimators for the treatment effect estimators. The methods and theoretical results are applied and checked in a series of simulation studies. The third part of this thesis is devoted to a comparison between propensity-score matching and multiple linear regression using simulation.

Download or read book Using a Two Staged Propensity Score Matching Strategy and Multilevel Modeling to Estimate Treatment Effects in a Multisite Observational Study written by Jordan H. Rickles and published by . This book was released on 2012 with total page 13 pages. Available in PDF, EPUB and Kindle. Book excerpt: The study is designed to demonstrate and test the utility of the proposed two-stage matching method compared to other analytic methods traditionally employed for multisite observational studies. More specifically, the study addresses the following research questions: (1) How do different specifications of the matching method influence covariate balance? (2) How do different specifications in the matching method influence inferences about treatment effect and effect heterogeneity? The different matching method specifications include differences in the propensity score model and whether a between-site match, within-site match, or two-stage matching process is used. The simulation results indicate that the two-stage matching method balances the desire for within-site covariate balance and the desire to retain as many treatment units in the analysis as possible. Relative to more straightforward matching methods, however, the two-stage matching method does not result in greater covariate balance nor less biased effect estimation. As a result, more straightforward methods that address the nested data structure--such as within-site matching or pooled matching with a random-intercept-and-slope propensity score model--might be preferable to the more complex two-stage matching method. These conclusions are based on a finite set of data generating conditions, with a small set of important confounders at both the unit and site level and a reasonable within-site sample size for matching. Future research should examine the performance of various propensity score model and matching methods under more extreme data conditions. (Contains 2 tables and 5 figures.).

Download or read book Evaluating the Performance of Propensity Score Matching Based Approaches in Individual patient Data Meta analysis written by Fatema Tuj Johara and published by . This book was released on 2019 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: "Individual-patient data meta-analysis IPD-MA is an increasingly popular approach because of its analytical benefits. On the other hand, IPD-MA of observational studies must overcome the problem of confounding, otherwise biased estimates of treatment effect may be obtained. One approach to reducing confounding bias could be the use of propensity score matching. Recently, several PSM-based approaches have been used for the analysis of clustered data. IPD-MA can be considered as two-stage clustered data (patients within studies) and PSM-based approaches can apply as well. However, there is a difference between two structures. In IPD-MA, treatment prevalence for single treatment may vary greatly, for example, 0 to 100% according to studies. Moreover, the extent of heterogeneity in meta-analysis may be greater than that typically encountered in clustered data.Concerning the statistical and analytical challenges of IPD meta-analysis, in this thesis, we considered the formulation of four PSM based approaches for the analysis of IPD-MA of observational studies. We investigated the performance of these approaches through a simulation study, which considers an IPD-MA that examines the success of different treatments for multiple-resistant drug tuberculosis (MDR-TB). The simulation has been varied according to treatment prevalence, heterogeneity between studies and pooled odds ratio.The performance of PSM based techniques have been investigated through bias, variance, coverage of 95% Wald confidence interval and statistical power of the pooled odds ratio estimates. This thesis provides a recommendation regarding which PSM based approaches could be used according to different treatment prevalence scenarios and various extent of heterogeneity between studies." --

Download or read book Performance of Parametric Vs Data Mining Methods for Estimating Propensity Scores with Multilevel Data written by Meng Fan and published by . This book was released on 2020 with total page 158 pages. Available in PDF, EPUB and Kindle. Book excerpt: There are several limitations in this study. First, this study did not consider varied correlation between covariates. Future research can be done to incorporate varied correlations among covariates. Second, balanced cluster size scenarios were created in this study. It is worth exploring the effect of the imbalance on the estimation of treatment effect. Third, this study included only propensity score weighting as the conditioning method. Future research can assess the performance of data mining approaches to estimate the propensity score using matching and stratification conditioning methods. Fourth, when using GBM to generate the propensity score in this study, only one algorithm specification was specified. Further research should include different algorithm specifications for GBM with multilevel data.

Download or read book Bayesian Latent Propensity Score Approach for Average Causal Effect Estimation Allowing Covariate Measurement Error written by Elande Baro and published by . This book was released on 2015 with total page 444 pages. Available in PDF, EPUB and Kindle. Book excerpt: Subclassification on the propensity score is a commonly-used analytic method for estimating the average causal effect of treatment on outcomes in observational studies. This method relies on nonconfoundedness assumption for valid inference on the average causal effect (ACE) estimation, i.e. no unobserved confounder and no covariate measurement error. However, many postmarketing studies rely on observational data where confounders (i.e. covariates) are subject to unobserved measurement error. It has been shown that using the naive propensity score methods ignoring such error will bias the ACE inference. Huang et al. extended the standard propensity score based causal framework to allow covariate measurement error using a latent propensity score, and developed a joint likelihood based approach for consistent ACE estimation using EM algorithm under finite mixture model of continuous outcomes. The numerical performance of the EM algorithm is not always ideal with increasing dimensions of unknown parameters. We extend this work to more flexible outcomes including binary/binomial outcomes using Bayesian MCMC algorithm. Theoretically, we derive the sufficient conditions for the identifiability of the proposed finite mixture models of binary/binomial outcomes. In terms of computation, a Bayesian estimation method is used to fit the latent propensity score model under finite mixture model format. Simulations studies often show superior performance of this newly developed Bayesian approach regarding bias, variance, and MSE criteria, compared to the existing EM algorithm and na ̈ıve approach (ignoring the error). We apply this method in post-market evaluation of the health impact of long-term breast pump usage on infant’s cumulative comorbidity index during first year, using the recently collected Infant Feeding Practice Study (II) data.

Download or read book Propensity Score Analysis written by Shenyang Guo and published by SAGE. This book was released on 2015 with total page 449 pages. Available in PDF, EPUB and Kindle. Book excerpt: Provides readers with a systematic review of the origins, history, and statistical foundations of Propensity Score Analysis (PSA) and illustrates how it can be used for solving evaluation and causal-inference problems.

Download or read book Propensity Score Adjustment in Multiple Group Observational Studies written by Erinn M. Hade and published by . This book was released on 2012 with total page 96 pages. Available in PDF, EPUB and Kindle. Book excerpt: Abstract: In medical and public health research, many studies are observational. In these studies, the treatment is not randomly assigned to participants. Therefore, the dif- ferences in outcomes between treatment groups could be due to imbalances of char- acteristics that are related to the outcome of interest prior to the treatment. Herein we investigate how we can use propensity scores, the conditional probability of re- ceiving treatment given the observed information, to make valid causal inference in observational studies. Theoretical results for the bias are given for linear response models that use the propensity score as a linear covariate. The bias depends on the relationship between the propensity score, the treatment indicator and the functional form of the covariate. Various methods for estimation of the treatment effect are explored. We show that the bias is influenced by the overlap in the distributions and functional forms of the covariates. When the distributions of the covariates have substantial overlap between treated and control groups, matching does well in terms of bias. In the second half of our work, we continue to investigate propensity score meth- ods for causal inference in observational studies, however our focus turns to studies with multiple groups. These methods are motivated by an example from the Pre- maturity Prevention Clinic at The Ohio State University. Our innovation relies on matching triplets of patients, which includes one patient from each of our groups of interest (those treated on-time, those with delayed treatment, and those who never were treated with 17P). Within each of these triplets, we attempt to balance pre- treatment characteristics by two matching techniques. We investigate two match- ing algorithms via simulation. Theese simulation studies found that a sub-optimal ii matching approach will in most circumstances provide better overall matches, than a nearest-neighbor approach. We implement our sub-optimal triplet matching for our motivating data and provide some conclusions about these data and future work with these methods.

Download or read book Robust Interval Estimation of a Treatment Effect in Observational Studies Using Propensity Score Matching written by Scott F. Kosten and published by . This book was released on 2010 with total page 236 pages. Available in PDF, EPUB and Kindle. Book excerpt: Estimating the treatment effect between a treatment group and a control group in an observational study is a challenging problem in statistics. Without random assignment of subjects, there are likely to be differences between the treatment group and control group on a set of baseline covariates. If one of these baseline covariates is correlated to the response variable, then the difference in sample means between the groups is likely to be a biased estimate of the true treatment effect. Propensity score matching has become an increasingly popular strategy for reducing bias in estimates of the treatment effect. This reduction in bias is accomplished by identifying a subset of the original control group, which is similar to the treatment group in terms of the measured baseline covariates. Our research focused on the development of a new procedure that combines propensity score matching and a rank-based analysis of the general linear model. Our procedure was compared to several others in a Monte Carlo simulation study. Overall, our procedure produced highly efficient and robust confidence intervals for a treatment effect in an observational study. In addition to the Monte Carlo simulation study, our procedure and several other propensity score matching techniques were used to analyze two real world datasets for the presence of a treatment effect.