UESD

The UESD is especially useful to analyze the causal effects of evnts, like for example terrorist attacks, that cannot be easily manipulated through experiments. Under certain assumptions, the UESD can provide estimates of the causal effects of different types of events on theoretically relevant outcomes. This capacity is based on three crucial features of this design. First, as-if random assignment to treatment and control groups increases the internal validity of the estimates as it can shield them from biases related to unobserved confounders or reverse causality. Second, the fact that the UESD relies on naturally occurring events provides a level of external validity that goes well beyond what controlled experiments can offer. Third, in contrast to most analyses of the impact of events that use longitudinal data or different surveys stacked around the date of an event, UESD studies employ a single survey to focus on the immediate period of time surrounding the event, which ensures that the impact of other macro-level confounders is less of a concern.

The absence of a true random assignment and the lack of control by the researchers pose challenges to identification that need to be addressed upfront if this strategy is to be used to substantiate causal claims. There are two main threats: ignorability and excludability. Simply put, ignorability implies that the respondents interviewed before and after the event are comparable. For this to be the case the moment at which respondents are interviewed during the fieldwork should be as-if random. Excludability implies that any difference in the outcome between the pre- and post-event contexts is the sole consequence of the event. That is, that the pre- and post-event contexts are comparable.

Reachability refers to how easy/difficult it is to interview a given survey respondent. The problem for the UESD is that some types of respondents are more elusive to survey researchers. Differences in reachability are related to respondents’ ease of contact and their inability or reluctance to take the survey. For example, in the case of landline telephone surveys those who are more often at home, like the elderly, are more reachable. In contrast those who work are less often at home and are more difficult to reach for an interview. In this case, those who are interviewed before the event will probably be different than those who are interviewed after the event. That is, the ignorability assumption will be violated.

The plausibility of the ignorability assumption can be assessed through balance tests that evaluate to what extent the pre and post-event samples are comparable, and by analyzing whether attrition patterns are different before and after the event. Moreover, detailed background information on the survey and the fieldwork is also crucial in order to evaluate the plausibility of this assumption. To try to address violations of the ignorability assumption researchers can narrow the bandwidth of fieldwork days considered around the event and/or use some form of covariate adjustment, paying special attention to covariates related to the reachability of respondents and variables related to the survey design (e.g. those used to establish survey quotas).

The plausibility of the excludability assumption can be assessed through: (i) a detailed analysis and description of the event and the circumstances surrounding it; (ii) the inspection of pre-existing time trends within the control group; (iii) placebo tests on other outcome variables; (iv) falsification tests based on other surveys or units, where an effect should not be observed.