Download or read book Deep Learning Based Causal Inference for Large Scale Combinatorial Experiments written by Zikun Ye and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Large-scale online platforms launch hundreds of randomized experiments (a.k.a. A/B tests) every day to iterate their operations and marketing strategies, while the combinations of these treatments are typically not exhaustively tested. It triggers an important question of both academic and practical interests: Without observing the outcomes of all treatment combinations, how to estimate the causal effect of any treatment combination and identify the optimal treatment combination? We develop a novel framework combining deep learning and doubly robust estimation to estimate the causal effect of any treatment combination for each user on the platform when observing only a small subset of treatment combinations. Our proposed framework (called debiased deep learning, DeDL) exploits Neyman orthogonality and combines interpretable and flexible structural layers in deep learning. We prove theoretically that this framework yields efficient, consistent, and asymptotically normal estimators under mild assumptions, thus allowing for identifying the best treatment combination when only observing a few combinations. To empirically validate our method, we then collaborate with a large-scale video-sharing platform and implement our framework for three experiments involving three treatments where each combination of treatments is tested. When only observing a subset of treatment combinations, our DeDL approach significantly outperforms other benchmarks to accurately estimate and infer the average treatment effect (ATE) of any treatment combination, and to identify the optimal treatment combination.

Download or read book ICT for Intelligent Systems written by Jyoti Choudrie and published by Springer Nature. This book was released on with total page 382 pages. Available in PDF, EPUB and Kindle. Book excerpt:

Download or read book Elements of Causal Inference written by Jonas Peters and published by MIT Press. This book was released on 2017-11-29 with total page 289 pages. Available in PDF, EPUB and Kindle. Book excerpt: A concise and self-contained introduction to causal inference, increasingly important in data science and machine learning. The mathematization of causality is a relatively recent development, and has become increasingly important in data science and machine learning. This book offers a self-contained and concise introduction to causal models and how to learn them from data. After explaining the need for causal models and discussing some of the principles underlying causal inference, the book teaches readers how to use causal models: how to compute intervention distributions, how to infer causal models from observational and interventional data, and how causal ideas could be exploited for classical machine learning problems. All of these topics are discussed first in terms of two variables and then in the more general multivariate case. The bivariate case turns out to be a particularly hard problem for causal learning because there are no conditional independences as used by classical methods for solving multivariate cases. The authors consider analyzing statistical asymmetries between cause and effect to be highly instructive, and they report on their decade of intensive research into this problem. The book is accessible to readers with a background in machine learning or statistics, and can be used in graduate courses or as a reference for researchers. The text includes code snippets that can be copied and pasted, exercises, and an appendix with a summary of the most important technical concepts.

Download or read book Causal Inference in Python written by Matheus Facure and published by "O'Reilly Media, Inc.". This book was released on 2023-07-14 with total page 428 pages. Available in PDF, EPUB and Kindle. Book excerpt: How many buyers will an additional dollar of online marketing bring in? Which customers will only buy when given a discount coupon? How do you establish an optimal pricing strategy? The best way to determine how the levers at our disposal affect the business metrics we want to drive is through causal inference. In this book, author Matheus Facure, senior data scientist at Nubank, explains the largely untapped potential of causal inference for estimating impacts and effects. Managers, data scientists, and business analysts will learn classical causal inference methods like randomized control trials (A/B tests), linear regression, propensity score, synthetic controls, and difference-in-differences. Each method is accompanied by an application in the industry to serve as a grounding example. With this book, you will: Learn how to use basic concepts of causal inference Frame a business problem as a causal inference problem Understand how bias gets in the way of causal inference Learn how causal effects can differ from person to person Use repeated observations of the same customers across time to adjust for biases Understand how causal effects differ across geographic locations Examine noncompliance bias and effect dilution

Download or read book Causality in a Social World written by Guanglei Hong and published by John Wiley & Sons. This book was released on 2015-08-17 with total page 443 pages. Available in PDF, EPUB and Kindle. Book excerpt: Causality in a Social World introduces innovative new statistical research and strategies for investigating moderated intervention effects, mediated intervention effects, and spill-over effects using experimental or quasi-experimental data. The book uses potential outcomes to define causal effects, explains and evaluates identification assumptions using application examples, and compares innovative statistical strategies with conventional analysis methods. Whilst highlighting the crucial role of good research design and the evaluation of assumptions required for identifying causal effects in the context of each application, the author demonstrates that improved statistical procedures will greatly enhance the empirical study of causal relationship theory. Applications focus on interventions designed to improve outcomes for participants who are embedded in social settings, including families, classrooms, schools, neighbourhoods, and workplaces.

Download or read book Using Latent Variable Models to Improve Causal Estimation written by Huseyin Oktay and published by . This book was released on 2018 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Estimating the causal effect of a treatment from data has been a key goal for a large number of studies in many domains. Traditionally, researchers use carefully designed randomized experiments for causal inference. However, such experiments can not only be costly in terms of time and money but also infeasible for some causal questions. To overcome these challenges, causal estimation methods from observational data have been developed by researchers from diverse disciplines and increasingly studies using such methods account for a large share in empirical work. Such growing interest has also brought together two arguably separate fields: machine learning and causal estimation, and this thesis also contributes to this intersection. Specifically, in observational data researchers have lack of control over the data generation process. This results in a fundamental challenge: the presence of confounder variables (i.e., variables that affect both treatment and outcome). Such variables, when not adjusted statistically, can result in biased causal estimates. When confounder variables are observed, many methods can be used to adjust for their effect. However, in most real world observational data sets, accurately measuring all potential confounder variables is far from feasible, hence important confounder variables are likely to remain unobserved. The central idea of this thesis is to explicitly account for unobserved confounders by inferring their values using a predictive model. This thesis presents three main contributions in the intersection of machine learning and causal estimation. First, we present one of the earliest application of causal estimation methods from social sciences to social media platforms to answer three causal questions. Second, we present a novel generative model for estimating ordinal variables with distant supervision. We also apply this model to data from US Twitter user population and discover variation in behavior among users from different age groups. Third, we characterize the behavior of an effect restoration model based on graphical models with theoretical analysis and simulation studies. We also apply this effect restoration model with predictive models to account for unobserved confounder variables.

Download or read book An Introduction to Causal Inference written by Judea Pearl and published by Createspace Independent Publishing Platform. This book was released on 2015 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This paper summarizes recent advances in causal inference and underscores the paradigmatic shifts that must be undertaken in moving from traditional statistical analysis to causal analysis of multivariate data. Special emphasis is placed on the assumptions that underly all causal inferences, the languages used in formulating those assumptions, the conditional nature of all causal and counterfactual claims, and the methods that have been developed for the assessment of such claims. These advances are illustrated using a general theory of causation based on the Structural Causal Model (SCM) described in Pearl (2000a), which subsumes and unifies other approaches to causation, and provides a coherent mathematical foundation for the analysis of causes and counterfactuals. In particular, the paper surveys the development of mathematical tools for inferring (from a combination of data and assumptions) answers to three types of causal queries: (1) queries about the effects of potential interventions, (also called "causal effects" or "policy evaluation") (2) queries about probabilities of counterfactuals, (including assessment of "regret," "attribution" or "causes of effects") and (3) queries about direct and indirect effects (also known as "mediation"). Finally, the paper defines the formal and conceptual relationships between the structural and potential-outcome frameworks and presents tools for a symbiotic analysis that uses the strong features of both. The tools are demonstrated in the analyses of mediation, causes of effects, and probabilities of causation. -- p. 1.

Download or read book Causal Inference with Non standard Experimental Designs written by Han Wu (Researcher in causal inference) and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: The past decades have seen a comprehensive body of research dedicated to causal inference in conventional experimental designs. However, as technological innovations continue to foster a rapid influx of data across numerous fields, the datasets derived often exhibit new structures that stem from unconventional designs. The thesis at hand is centered around the development of methods for conducting causal inference, particularly when the design deviates from the standard, thereby making conventional methods inapplicable. Chapter 2 delves into the regression discontinuity design in cases where the running variable is a noisy measurement of a latent variable. We propose a novel design-based approach for estimation and inference. This approach proves effective when applied to a broad array of widely-used estimands. Chapter 3 explores adaptive experimentation in the context of delayed feedback. In subchapter 3.1, we extend Thompson sampling to the proportional hazard model and develop a method capable of overcoming challenges associated with vaccine trials. Subsequently, in subchapter 3.2, we study the behavior of Thompson sampling when delays are unrestricted, providing theoretical regret bounds and conducting extensive experiments. Chapter 4 investigates policy learning in scenarios involving multiple treatments or multiple outcomes. In subchapter 4.1, we propose methods for evaluating policies when cost constraints accompany multiple treatments. In subchapter 4.2, we introduce a personalized experimentation system that can learn interpretable policies from experimental data and is scalable for big datasets.

Download or read book Observation and Experiment written by Paul R. Rosenbaum and published by . This book was released on 2017 with total page 374 pages. Available in PDF, EPUB and Kindle. Book excerpt: Cover -- Contents -- Preface -- Reading Options -- List of Examples -- Part I. Randomized Experiments -- 1. A Randomized Trial -- 2. Structure -- 3. Causal Inference in Randomized Experiments -- 4. Irrationality and Polio -- Part II. Observational Studies -- 5. Between Observational Studies and Experiments -- 6. Natural Experiments -- 7. Elaborate Theories -- 8. Quasi-experimental Devices -- 9. Sensitivity to Bias -- 10. Design Sensitivity -- 11. Matching Techniques -- 12. Biases from General Dispositions -- 13. Instruments -- 14. Conclusion -- Appendix: Bibliographic Remarks -- Notes -- Glossary: Notation and Technical Terms -- Suggestions for Further Reading -- Acknowledgments -- Index

Download or read book Essays in Causal Inference and Experimental Design written by Orville Mondal and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation is comprised of two chapters the first of which is concerned with how to address a prevalent issue when analyzing data from randomized experiment with human participants, and the second of which investigates aspects of experimental design. In Chapter 1 I study the problem of identifying the causal effect of an experimental treatment when the experiment suffers from non-compliance. In particular, I consider the identifying power of information collected from non-complying participants after the completion of the treatment phase. Follow up surveys often ask study participants why they chose not to accept an offer of treatment despite being assigned to it, and answers to such questions offer insights into the decision process by which agents choose to comply with their assigned treatment status. I propose a model that rationalizes an agent's compliance decision. Based on this model, I characterize the set of values for the average treatment effect that are conformable with data observed in a randomized trial. This model and the implied set of identified values for the average treatment effect rely on the availability of follow up surveys that ask agents why they chose to not comply. This underscores the importance of following up with non-complying agents since the model often leads to substantially tighter identified sets for the average treatment effect than what is possible without this information. I apply the proposed model to data from the Job Training Partnership Act Study to estimate identified sets for the average treatment effect for a number of employment outcomes. Chapter 2 considers the problem of where to introduce a new policy. Field experiments can inform where to implement policies, but in practice there may be hundreds of candidate sites being considered and concerns about external validity. We develop a quasi-Bayesian approach to selecting a small number of sites in which to run new experiments to inform policy choices across a wide set of locations. The approach develops a prior specification for the joint distribution of site-level average treatment effects based on a microeconometric structural model, while also allowing for heterogeneity across sites that is not captured by the model. We apply the approach to choices over rolling out a mobile money innovation in 41 migration corridors in Bangladesh, 60 in Pakistan, and 740 in India. Learning is optimized when the experiments take place in locations informative for the largest number of sites where suitability of the policy is a priori uncertain. Learning and expected welfare improve relative to experimentation based on plausible rules of thumb.

Download or read book Causal Discovery of Adverse Drug Events in Observational Data written by Aubrey Barnard and published by . This book was released on 2019 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Automatic causal discovery without experiments offers to accelerate scientific investigation and knowledge acquisition, for example, by searching databases of electronic health records to discover the unknown effects of drugs. However, effective causal discovery requires methods that control for confounders and that scale to large data sets which have the power to support or refute causal hypotheses. Accordingly, this dissertation first introduces a method for efficiently learning formal structural causal models of medical histories via parameter learning in log-linear temporal Markov networks. Such models work well when all of the effects of interest are already defined and measured, but it might not be the case that all possible effects are suspected beforehand, especially when considering the adverse effects of drugs. Therefore, this dissertation next develops machine learning methods for causal discovery, including differential classification and temporal inverse probability weighting, that hypothesize likely causal effects while analyzing controlled observational studies. Applying all of these methods to causal modeling and finding adverse drug effects in synthetic and real-world electronic health records demonstrates their ability to accurately discover causal effects despite the irregularity, noise, and sparsity of such data. This dissertation thus establishes (1) that scalable, causal methods discover causal effects more accurately than methods that ignore causality, do not scale to large databases, or are not robust to the messiness of medical data, and (2) that methods that hypothesize effects improve genuine causal discovery by avoiding the limitations of human bias. In summary, the methods herein distinguish themselves by bridging machine learning and epidemiology: they bring causal inference and observational studies to machine learning, and they apply learning techniques and formal causal models to tasks in epidemiology. By integrating multiple approaches to causality, these methods achieve a wider perspective that overcomes the limitations of the individual perspectives, and leads to new methods for automatic causal discovery from observational data.

Download or read book An Introduction to Causal Inference written by and published by . This book was released on 2009 with total page 69 pages. Available in PDF, EPUB and Kindle. Book excerpt: This paper summarizes recent advances in causal inference and underscores the paradigmatic shifts that must be undertaken in moving from traditional statistical analysis to causal analysis of multivariate data. Special emphasis is placed on the assumptions that underly all causal inferences, the languages used in formulating those assumptions, the conditional nature of all causal and counterfactual claims, and the methods that have been developed for the assessment of such claims. These advances are illustrated using a general theory of causation based on the Structural Causal Model (SCM) described in Pearl (2000a), which subsumes and unifies other approaches to causation, and provides a coherent mathematical foundation for the analysis of causes and counterfactuals. In particular, the paper surveys the development of mathematical tools for inferring (from a combination of data and assumptions) answers to three types of causal queries: (1) queries about the effects of potential interventions, (also called "causal effects" or "policy evaluation") (2) queries about probabilities of counterfactuals, (including assessment of "regret," "attribution" or "causes of effects") and (3) queries about direct and indirect effects (also known as "mediation"). Finally, the paper defines the formal and conceptual relationships between the structural and potential-outcome frameworks and presents tools for a symbiotic analysis that uses the strong features of both.

Download or read book Causal Inference for High Stakes Decisions written by Harsh J. Parikh and published by . This book was released on 2023 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Causal inference methods are commonly used across domains to aid high-stakes decision-making. The validity of causal studies often relies on strong assumptions that might not be realistic in high-stakes scenarios. Inferences based on incorrect assumptions frequently result in sub-optimal decisions with high penalties and long-term consequences. Unlike prediction or machine learning methods, it is particularly challenging to evaluate the performance of causal methods using just the observed data because the ground truth causal effects are missing for all units. My research presents frameworks to enable validation of causal inference methods in one of the following three ways: (i) auditing the estimation procedure by a domain expert, (ii) studying the performance using synthetic data, and (iii) using placebo tests to identify biases. This work enables decision-makers to reason about the validity of the estimation procedure by thinking carefully about the underlying assumptions. Our Learning-to-Match framework is an auditable-and-accurate approach that learns an optimal distance metric for estimating heterogeneous treatment effects. We augment Learning-to-Match framework with pharmacological mechanistic knowledge to study the long-term effects of untreated seizure-like brain activities in critically ill patients. Here, the auditability of the estimator allowed neurologists to qualitatively validate the analysis via a chart-review. We also propose Credence, a synthetic data based framework to validate causal inference methods. Credence simulates data that is stochastically indistinguishable from the observed data while allowing for user-designed treatment effects and selection biases. We demonstrate Credence's ability to accurately assess the relative performance of causal estimation techniques in an extensive simulation study and two real-world data applications. We also discuss an approach to combines experimental and observational studies. Our approach provides a principled approach to test for the violations of key assumptions and estimate causal effects (Chapter 5).

Download or read book Essays in Causal Inference written by Michael Pollmann and published by . This book was released on 2022 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: This dissertation explores the estimation of causal effects in settings with non-standard data. In the first chapter, the treatments are not directly assigned to outcome units but instead occur in the same geographic space. In the second chapter, responses to hypothetical questions describing both the treated and control state are used to learn about the effects of a treatment on real behavior (outcomes). In the third chapter, treatments are assigned according to a randomized experiment but outcomes are heavy-tailed, such that semiparametric approaches are useful to improve efficiency and robustness. The first chapter considers settings where the treatments causing the effects of interest are not directly associated with specific units for which we measure outcomes, but rather occur in the same geographic space. Many events and policies (treatments), such as opening of businesses, building of hospitals, and sources of pollution, occur at specific spatial locations, with researchers interested in their effects on nearby individuals or businesses (outcome units). However, the existing treatment effects literature primarily considers treatments that could experimentally be assigned directly at the level of the outcome units, potentially with spillover effects. I approach the spatial treatment setting from a similar experimental perspective: What ideal experiment would we design to estimate the causal effects of spatial treatments? This perspective motivates a comparison between individuals near realized treatment locations and individuals near counterfactual (unrealized) candidate locations, which is distinct from current empirical practice. I derive standard errors based on this design-based perspective that are straightforward to compute irrespective of spatial correlations in outcomes. Furthermore, I propose machine learning methods to find counterfactual candidate locations and show how to apply the proposed methods on observational data. I study the causal effects of grocery stores on foot traffic to nearby businesses during COVID-19 shelter-in-place policies. I find a substantial positive effect at a very short distance. Correctly accounting for possible effect "interference" between grocery stores located close to one another is of first order importance when calculating standard errors in this application. The second chapter is co-authored with B. Douglas Bernheim, Daniel Björkegren, and Jeffrey Naecker. We explore methods for inferring the causal effects of treatments on choices by combining data on real choices with hypothetical evaluations. We propose a class of estimators, identify conditions under which they yield consistent estimates, and derive their asymptotic distributions. The approach is applicable in settings where standard methods cannot be used (e.g., due to the absence of helpful instruments, or because the treatment has not been implemented). It can recover heterogeneous treatment effects more comprehensively, and can improve precision. We provide proof of concept using data generated in a laboratory experiment and through a field application. The final chapter is co-authored with Susan Athey, Peter J. Bickel, Aiyou Chen, and Guido W. Imbens. We develop new semiparametric methods for estimating treatment effects. We focus on a setting where the outcome distributions may be heavy-tailed, where treatment effects are small, where sample sizes are large and where assignment is completely random. This setting is of particular interest in recent experimentation in tech companies. We propose using parametric models for the treatment effects, as opposed to parametric models for the full outcome distributions. This leads to semiparametric models for the outcome distributions. We derive the semiparametric efficiency bound for this setting, and propose efficient estimators. In the case with a constant treatment effect one of the proposed estimators has an interesting interpretation as a weighted average of quantile treatment effects, with the weights proportional to (minus) the second derivative of the log of the density of the potential outcomes. Our analysis also results in an extension of Huber's model and trimmed mean to include asymmetry and a simplified condition on linear combinations of order statistics, which may be of independent interest.

Download or read book Sharing Economy written by Ming Hu and published by Springer. This book was released on 2019-01-11 with total page 536 pages. Available in PDF, EPUB and Kindle. Book excerpt: This edited book examines the challenges and opportunities arising from today’s sharing economy from an operations management perspective. Individual chapter authors present state-of-the-art research that examines the general impact of sharing economy on production and consumption; the intermediary role of a sharing platform; crowdsourcing management; and context-based operational problems. Sharing economy refers to a market model that enables and facilitates the sharing of access to goods and services. For example, Uber allows riders to share a car. Airbnb allows homeowners to share their extra rooms with renters. Groupon crowdsources demands, enabling customers to share the benefit of discounted goods and services, whereas Kickstarter crowdsources funds, enabling backers to fund a project jointly. Unlike the classic supply chain settings in which a firm makes inventory and supply decisions, in sharing economy, supply is crowdsourced and can be modulated by a platform. The matching-supply-with-demand process in a sharing economy requires novel perspectives and tools to address challenges and identify opportunities. The book is comprised of 20 chapters that are divided into four parts. The first part explores the general impact of sharing economy on the production, consumption, and society. The second part explores the intermediary role of a sharing platform that matches crowdsourced supply with demand. The third part investigates the crowdsourcing management on a sharing platform, and the fourth part is dedicated to context-based operational problems of popular sharing economy applications. “While sharing economy is becoming omnipresence, the operations management (OM) research community has begun to explore and examine different business models in the transportation, healthcare, financial, accommodation, and sourcing sectors. This book presents a collection of the state-of-the-art research work conducted by a group of world-leading OM researchers in this area. Not only does this book cover a wide range of business models arising from the sharing economy, but it also showcases different modeling frameworks and research methods that cannot be missed. Ultimately, this book is a tour de force – informative and insightful!” Christopher S. Tang Distinguished Professor and Edward Carter Chair in Business Administration UCLA Anderson School of Management

Download or read book Demand Prediction in Retail written by Maxime C. Cohen and published by Springer Nature. This book was released on 2022-01-01 with total page 166 pages. Available in PDF, EPUB and Kindle. Book excerpt: From data collection to evaluation and visualization of prediction results, this book provides a comprehensive overview of the process of predicting demand for retailers. Each step is illustrated with the relevant code and implementation details to demystify how historical data can be leveraged to predict future demand. The tools and methods presented can be applied to most retail settings, both online and brick-and-mortar, such as fashion, electronics, groceries, and furniture. This book is intended to help students in business analytics and data scientists better master how to leverage data for predicting demand in retail applications. It can also be used as a guide for supply chain practitioners who are interested in predicting demand. It enables readers to understand how to leverage data to predict future demand, how to clean and pre-process the data to make it suitable for predictive analytics, what the common caveats are in terms of implementation and how to assess prediction accuracy.

Download or read book Deep Learning in Biology and Medicine written by Davide Bacciu and published by World Scientific Publishing Europe Limited. This book was released on 2021 with total page 0 pages. Available in PDF, EPUB and Kindle. Book excerpt: Biology, medicine and biochemistry have become data-centric fields for which Deep Learning methods are delivering groundbreaking results. Addressing high impact challenges, Deep Learning in Biology and Medicine provides an accessible and organic collection of Deep Learning essays on bioinformatics and medicine. It caters for a wide readership, ranging from machine learning practitioners and data scientists seeking methodological knowledge to address biomedical applications, to life science specialists in search of a gentle reference for advanced data analytics.With contributions from internationally renowned experts, the book covers foundational methodologies in a wide spectrum of life sciences applications, including electronic health record processing, diagnostic imaging, text processing, as well as omics-data processing. This survey of consolidated problems is complemented by a selection of advanced applications, including cheminformatics and biomedical interaction network analysis. A modern and mindful approach to the use of data-driven methodologies in the life sciences also requires careful consideration of the associated societal, ethical, legal and transparency challenges, which are covered in the concluding chapters of this book.