Quasi-Experiments II


Winter 2023

Course surveys due April 12, 11:59 PM


  • Final projects due April 21

  • Groups need to meet with instructor one more time before April 19 (Otherwise your group meeting grade is F)

  • Extra office hour times April 13-19

  • Every group member needs to be in at least one group meeting to receive the group meeting grade

Last time

  • Quasi-experiments: Observational answer strategies for causal inquiries
Data strategy
Inquiry Observational Experimental
Descriptive Sample survey List experiment
Causal Quasi-experiment Survey/field experiment
  • RDD as an example of quasi-experiment

  • Today: Difference-in-differences as another common example in public policy

What did you learn this semester?

Where to go from here?

Go back to foundations

  • Probability and statistics
  • Philosophy of science
  • Research design
  • R programming

Where to go from here?

Further learning

  • Programming in Python, Julia
  • Survey design
  • Program evaluation
  • Science of science

Where to go from here?

Careers & fields

  • Data science, computer science, statistics

  • Computational/quantitative social science

  • Econometrics

  • Evidence-informed policy

  • Public administration

  • Business, marketing



  • Critiques to causal claims from observational data
  1. Reverse causation

  2. Omitted variable bias

  3. Selection bias

  • Need creativity to rule these out

Difference-in-differences design

  • At least two groups or conditions (treatment,control)

  • At least two time periods (pre- and post-treatment)

  • Once treated, units stay on

  • We accept that selection bias is unavoidable

  • But comparing before-after changes between groups allows us to calculate treatment effect

Diff-in-diffs estimator

Group Before After
Treatment A B
Control C D

\[ \widehat{ATE} = [\text{Mean}(B) - \text{Mean}(A)] - [\text{Mean}(D) - \text{Mean}(C)] \]

Diff-in-diffs estimator

Group Before After
Treatment A B
Control C D

\[ \widehat{ATE} = \underbrace{[\text{Mean}(B) - \text{Mean}(A)]}_\text{Difference} - \underbrace{[\text{Mean}(D) - \text{Mean}(C)]}_\text{Difference} \]

Diff-in-diffs estimator

Group Before After
Treatment A B
Control C D

\[ \widehat{ATE} = \underbrace{\underbrace{[\text{Mean}(B) - \text{Mean}(A)]}_\text{Difference} - \underbrace{[\text{Mean}(D) - \text{Mean}(C)]}_\text{Difference}}_\text{Difference in differences} \]


Parallel trends

  • Treatment and control may have different values before treatment

  • Absent treatment, the treatment group would have changed like the control group

  • This is equivalent to claiming that treatment and control, while different, follow a similar trajectory

  • Ideally, you justify by observing the outcome over many pre-treatment periods

Variants of the design

  • Many groups, treatments, time periods

  • Increasingly common: Units become treated at different time periods

  • Example: Policy adopted by cities over a time period

  • This makes similar to a staggered adoption design

  • But things get very complicated without randomization

Multiple treatment periods


Leininger et al (2023)

Temporary disenfranchisement in Germany

  • Discrepancies of minimum voting age across elections (municipal, state, national)

  • 16-17 year olds in Schleswig-Holstein can vote in local but not national elections

  • Temporary disenfranchisement may push voters away from democracy

Research design

  • Outcomes: Survey questions about internal/external efficacy, satisfaction with democracy, political interest


Thank you!

Break time!