Randomized Experiments in International trade August 2018 Amit Khandelwal Columbia Business school
Randomized Experiments in International Trade August 2018 Amit Khandelwal Columbia Business School 1
Introduction RCTs are uncommon in international trade research but common in some fields, like development and labor economics Most recent BREAD conference featured 8 papers, 7 were RCTs In this talk, I'll present two RCTs conducted on export-oriented firms, with the following goals Why RCTs are a useful toolkit for research in international trade? What we can learn(and what we cannot hope to learn)? Practical and logistical challenges of implementing RCTs on firms will not discuss econometric/statistical issues in designing experiments See duflo et al 2007 for an extensive and highly useful discussion of these Issues
Introduction • RCTs are uncommon in international trade research, but common in some fields, like development and labor economics • Most recent BREAD conference featured 8 papers, 7 were RCTs • In this talk, I’ll present two RCTs conducted on export-oriented firms, with the following goals: – Why RCTs are a useful toolkit for research in international trade? – What we can learn (and what we cannot hope to learn)? – Practical and logistical challenges of implementing RCTs on firms • (I will not discuss econometric/statistical issues in designing experiments. See Duflo et al 2007 for an extensive and highly useful discussion of these issues) 2
Introduction The big empirical questions in trade What are the gains from trade? Who gains /loses from international trade? What are the mechanisms of the adjustment process? Remain active areas of work because they are difficult to study Trade policy is an endogenous outcome of lobbying and political economy Trade barriers look similar to technology shocks, and often correlated (i. e. innovations in logistics is both a technology shock and a shock to offshoring Inherent data limitations incomplete admin data, lack of price data, cannot directly observe technology use, and the general coarseness of data
Introduction • The big empirical questions in trade – What are the gains from trade? – Who gains/loses from international trade? – What are the mechanisms of the adjustment process? • Remain active areas of work because they are difficult to study – Trade policy is an endogenous outcome of lobbying and political economy – Trade barriers look similar to technology shocks, and often correlated (i.e., innovations in logistics is both a technology shock and a shock to offshoring) – Inherent data limitations • incomplete admin data, lack of price data, cannot directly observe technology use, and the general coarseness of data 3
Progress with RCTs RCTs can make progress on these questions in two key ways Causal inference Identified moments Nakamura Steinsson 18 Additionally, RCTs are almost always paired with primary data collection Allow researchers to tailor data collection to match outcomes of interest in models
Progress with RCTs • RCTs can make progress on these questions in two key ways: – Causal inference – “Identified moments” Nakamura & Steinsson 18 • Additionally, RCTs are almost always paired with primary data collection – Allow researchers to tailor data collection to match outcomes of interest in models
Causal Inference (in brief What does randomization do? Consider a linear model =BT+∑7 Bi is the treatment effect on firm i Consider treated firms=1)and control firms i=O), and subtract averages 7=32+∑(2- b is the average treatment effect Randomization guarantees the second term holds in expectation
Causal Inference (in brief!) • What does randomization do? • Consider a linear model – is the treatment effect on firm i • Consider treated firms ( ) and control firms ( ), and subtract averages • is the average treatment effect • Randomization guarantees the second term holds in expectation 5
RCTs and moments Long tradition in international trade research of iterating between statistics uncovered from micro data and developing ge models Simple Moments Nakamura& Steinsson 18 Unconditional means variances Firm heterogeneity- trading firms differ substantially from domestic oriented firms(Bernard and Jensen 95) Identified moments A target statistic obtained as a response to an identified structural shock (i.e, causal effect)Nakamura& Steinsson 18 Local labor market effects Trade shocks affect the performance of local labor markets(Topalova 10, Kovak 13, Autor et al 13, Caliendo et al 18, Adao et al 18) RCTs can provide moments that useful for discriminating between models and mechanisms
RCTs and Moments • Long tradition in international trade research of iterating between statistics uncovered from micro data and developing GE models • Simple Moments Nakamura & Steinsson 18 – Unconditional means, variances – Firm heterogeneity - trading firms differ substantially from domesticoriented firms (Bernard and Jensen 95) • Identified Moments – “A target statistic obtained as a response to an identified structural shock (i.e., causal effect)” Nakamura & Steinsson 18 – Local labor market effects • Trade shocks affect the performance of local labor markets (Topalova 10, Kovak 13, Autor et al 13, Caliendo et al 18, Adao et al 18) • RCTs can provide moments that useful for discriminating between models and mechanisms 6
Limits of rcts Balance between treatment and control groups occurs only in expectation Sample is a trial sample, rarely a true random sample from population Most rcts in economics are still not double blinded Stable Unit Treatment Value Assumption ( SUTVA) Average treatment effects only identified if no spillovers GE Spillovers are important features of key trade insights (e. g. Eaton and Kortum 02: Hornbeck donaldson 16 External validity Good RCTs work hard to generate externally-valid research designs Can be achieved over time through many studies
Limits of RCTs • Balance between treatment and control groups occurs only in expectation – Sample is a trial sample, rarely a true random sample from population • Most RCTs in economics are still not double blinded • Stable Unit Treatment Value Assumption (SUTVA) – Average treatment effects only identified if no spillovers – GE/Spillovers are important features of key trade insights (e.g., Eaton and Kortum 02; Hornbeck Donaldson 16) • External validity – Good RCTs work hard to generate externally-valid research designs – Can be achieved over time through many studies 7
Road Map Impact of Exporting on Firm Performance: Evidence from a Randomized Trial"(D. Atkin, A Osman), Quarterly Journalof Economics, 2017 Organizational Barriers to Technology Adoption: Evidence from Soccer-ball Producers in Pakistan"(D. Atkin, A Chaudhry S Chaudry and e. verhoogen) Quarterly ournal of EconomicS, 2017
Road Map • “Impact of Exporting on Firm Performance: Evidence from a Randomized Trial” (D. Atkin, A. Osman), Quarterly Journal of Economics, 2017 • “Organizational Barriers to Technology Adoption: Evidence from Soccer-ball Producers in Pakistan” (D. Atkin, A. Chaudhry, S. Chaudry and E. Verhoogen), Quarterly Journal of Economics, 2017 8
Road Map Impact of Exporting on Firm Performance: Evidence from a Randomized Trial"(D. Atkin, A Osman), Quarterly Journal of Economics, 2017 Organizational Barriers to Technology Adoption: Evidence from Soccer-ball Producers in Pakistan"(D. Atkin, A Chaudhry S Chaudry and e. verhoogen) Quarterly ournal of EconomicS, 2017
Road Map • “Impact of Exporting on Firm Performance: Evidence from a Randomized Trial” (D. Atkin, A. Osman), Quarterly Journal of Economics, 2017 • “Organizational Barriers to Technology Adoption: Evidence from Soccer-ball Producers in Pakistan” (D. Atkin, A. Chaudhry, S. Chaudry and E. Verhoogen), Quarterly Journal of Economics, 2017 9
Motivation Large differences in productivity between developed and developing world (Hall Jones 1999, Bloom & Van Reenan 2007 10
Motivation • Large differences in productivity between developed and developing world (Hall & Jones 1999, Bloom & Van Reenan 2007) 10
