Natural Experiments: Design Elements for Optimal Causal Inference Without Randomization

Most changes in laws and regulations affecting population health represent natural experiments, where scientists do not control when and where they are implemented, and thus cannot randomly assign the legal “treatments” to some and not to others.

Many research design elements can be incorporated in evaluations of public health laws to produce accurate estimates of the size of a law’s effect with high levels of confidence that an observed effect is caused by the law. Combining design elements produces the strongest possible evidence on whether a law caused the hypothesized effect and magnitude of that effect. Well-designed studies of public health laws in natural real-world settings facilitate diffusion of effective regulatory strategies, producing significant reductions in population burdens of disease, injury and death.

In this paper, readers will learn to

  • Understand advantages of time-series data, with many repeated observations before and after a change in law, for evaluating the law’s effects.
  • Create a nested multiple comparison group study design for evaluating health effects of a law.
  • Combine several design elements in a single study to strengthen causal inference.
  • Identify resources for further study of legal epidemiology methods. 

Download the paper

Authors:

  • Alexander C. Wagenaar, PhD, Emory University
  • Kelli A. Komro, MPH, PhD, Emory University