Skip to main content

What am I watching? CAFE University: An Introduction to Quasi-experimental Causal Inference Methods for Health and Extreme Weather Research

Link to article

If you’re interested in contributing a short What Am I Reading post, we’d love to hear from you! Email us at cache@colorado.edu

Written by Jenna Tipaldo, CUNY Institute for Demographic Research 

In the webinar “CAFE University: An Introduction to Quasi-experimental Causal Inference Methods for Health and Extreme Weather Research” hosted by CAFE RCC, Professor Rachel Nethery makes the case for the use of quasi-experimental methods in studies of the impacts of extreme weather events.  This webinar focused on the use of two methods, which are used throughout the social and health sciences (Columbia University Mailman School of Public Health; Krajewski & Hudgens, 2024) to isolate the impact of extreme weather on health outcomes: difference-in difference and synthetic control approaches. 

Quasi-experimental methods use the language of causation (i.e., studies ascribing causation) to assess events that might not feasibly or ethically be randomized. If an experiment were hypothetically possible, an exposed group would be considered as ‘treated’ and compared to an ‘untreated’ group – those without exposure – to estimate the counterfactual scenario: What would have happened in the absence of the treatment?  Quasi-experimental methods aim to mimic this set-up using observational data. In extreme weather research, a ‘group’ or ‘unit’ may refer to geographic areas, such as counties or Census blocks, or individuals that were or weren’t impacted by a hazard, such as a storm, or hazard-related impacts, such as power outages. In the language of causation, a unit would be considered ‘treated’ if it was exposed and ‘untreated’ if it were not. While study designs that simply compare areas before and after an event or exposed and unexposed areas may fail to consider time trends or confounding differences between areas, quasi-experimental methods can provide a stronger case for causation by mitigating such sources of bias.  

Difference-in-difference analyses are useful for observational data where it is reasonable to assume that trends in two groups move in parallel, allowing for the estimation of a counterfactual in a treated group based on observation of an untreated group (see Columbia University Mailman School of Public Health’s site which links many readings, including applications in the health sciences). Regarding impacted areas, difference-in-difference methods are best suited for many treated units, such as if a storm impacted many counties. Difference-in-difference analyses require a few pre-treatment observations to help assess the plausibility of parallel trends. 

Synthetic control analyses are similarly used with observational data and are useful when there are only a few treated units, such as if only one or two counties were impacted by a flood. This method is desirable when control units that are relatively similar to the treated units can be identified and used to create construct a ‘synthetic control’, such as by constructing a weighted average of controls, that stands in for the counterfactual to the treated unit(s). While synthetic control methods do not require parallel trends, they do require many more observations in the pre-treatment period to assess how well the ‘synthetic control’ approximates the treated unit pre-treatment.  

The choice of these methods depends on the situation and data at hand. For example, both are generally good for the study of discrete events such as hurricanes or heatwaves, although not as well-equipped to measure the impacts of continuous exposures such as air pollution or temperature. This is because these methods are best suited to assess differences in an “after” time period based on trends in a “before” time period, which  helps establish the treated and control units as having similar trends. For ideas regarding research designs to study continuous exposures, see Professor Kai Chen’s CACHE Seminar).  

The webinar discusses spillover effects as a concern in choosing control units for comparison.  Spillover happens if individuals from impacted areas – treatment areas — move to nearby areas – control areas, thus muddling the observable effect. Displaced people may stay nearby for reasons such as staying near one’s community, job, and home to supervise rebuilding (Rhodes & Besbris, 2022). These issues may be amplified depending on the analytical spatial unit; for example, impacted and displaced persons may be more likely to remain within their county but less so their Census block or tract.  It is also important to consider the scale and dependencies of the event – i.e., if impacts are widespread rather than highly localized, such as if disruption of infrastructure such as major roads or transit in one area impacts other nearby areas. 

The webinar notes that it is not a complete review of causal methods, nor data types. Some other quasi-experimental methods include interrupted time series methods (Lopez Bernal et al., 2018) and augmented synthetic control methods (Krajewski & Hudgens, 2024). Regarding types of data, the webinar focused its coverage on data from administrative units, for example mortality in hurricane-affected counties. Difference-in-difference methods can be used on individual-level data as well (Columbia University Mailman School of Public Health). Individual-level data from surveys that can be used to study demographic and health outcomes and repurposed to study disaster impacts (Fussell, Burrows, & Sastry, 2025). Beyond containing elements of quasi-experimental research as described in the webinar (e.g., an exposed and unexposed comparison group, observations before and after an event), large population-representative survey samples allow for generalization and subgroup comparisons. Another advantage of using individual-level data is that the research can avoid the ecological fallacy and follow individuals over time, including if they move (such as if displaced due to a disaster).  

As literature on disaster impacts trends toward quasi-experimental work, which informs assessments of causation, there remains an important place for descriptive analysis of data (Duncan, 2008). For example, the U.S. population is aging, which means that in the future, a higher number and proportion of people are expected to face challenges such as chronic health conditions, that can influence vulnerability to disasters and environmental exposures such as extreme heat. Descriptions, including rates, prevalences, and spatial distributions, will also be necessary to inform policy and future planning, and are complementary and powerful tools especially when used alongside quasi-experimental studies. 

References 

  • Fussell, E., Burrows, K., & Sastry, N. (2025). Learning From Natural Experiments to Accelerate Demographic Research on Climate‐Related Threats to Human Populations. WIREs Climate Change, 16(6), e70031. https://doi.org/10.1002/wcc.70031 

  • Krajewski, T., & Hudgens, M. The augmented synthetic control method in public health and biomedical research. Statistical Methods in Medical Research. 2024;33(3):376-391. https://doi.org/10.1177/09622802231224638    

  • Lopez Bernal, J., Cummins, S., & Gasparrini, A. (2018). The use of controls in interrupted time series studies of public health interventions. International Journal of Epidemiology, 47(6), 2082–2093. https://doi.org/10.1093/ije/dyy135  

  • Rhodes, A., & Besbris, M. (2022). Soaking the middle class: Suburban inequality and recovery from disaster (First). Russell Sage Foundation.