Quasi-Experiments II
POLSCI 4SS3
Winter 2023
Course surveys due April 12, 11:59 PM
Announcements
Final projects due April 21
Groups need to meet with instructor one more time before April 19
(Otherwise your group meeting grade is F)Extra office hour times April 13-19
Every group member needs to be in at least one group meeting to receive the group meeting grade
Last time
- Quasi-experiments: Observational answer strategies for causal inquiries
Data strategy |
||
|---|---|---|
| Inquiry | Observational | Experimental |
| Descriptive | Sample survey | List experiment |
| Causal | Quasi-experiment | Survey/field experiment |
RDD as an example of quasi-experiment
Today: Difference-in-differences as another common example in public policy
What did you learn this semester?
Where to go from here?
Go back to foundations
- Probability and statistics
- Philosophy of science
- Research design
- R programming
Where to go from here?
Further learning
- Programming in Python, Julia
- Survey design
- Program evaluation
- Science of science
Where to go from here?
Careers & fields
Data science, computer science, statistics
Computational/quantitative social science
Econometrics
Evidence-informed policy
Public administration
Business, marketing
Difference-in-differences
Reminder
- Critiques to causal claims from observational data
Reverse causation
Omitted variable bias
Selection bias
- Need creativity to rule these out
Difference-in-differences design
At least two groups or conditions
(treatment,control)At least two time periods
(pre- and post-treatment)Once treated, units stay on
We accept that selection bias is unavoidable
But comparing before-after changes between groups allows us to calculate treatment effect
Diff-in-diffs estimator
Timing |
||
|---|---|---|
| Group | Before | After |
| Treatment | A | B |
| Control | C | D |
\[ \widehat{ATE} = [\text{Mean}(B) - \text{Mean}(A)] - [\text{Mean}(D) - \text{Mean}(C)] \]
Diff-in-diffs estimator
Timing |
||
|---|---|---|
| Group | Before | After |
| Treatment | A | B |
| Control | C | D |
\[ \widehat{ATE} = \underbrace{[\text{Mean}(B) - \text{Mean}(A)]}_\text{Difference} - \underbrace{[\text{Mean}(D) - \text{Mean}(C)]}_\text{Difference} \]
Diff-in-diffs estimator
Timing |
||
|---|---|---|
| Group | Before | After |
| Treatment | A | B |
| Control | C | D |
\[ \widehat{ATE} = \underbrace{\underbrace{[\text{Mean}(B) - \text{Mean}(A)]}_\text{Difference} - \underbrace{[\text{Mean}(D) - \text{Mean}(C)]}_\text{Difference}}_\text{Difference in differences} \]
Assumption
Parallel trends
Treatment and control may have different values before treatment
Absent treatment, the treatment group would have changed like the control group
This is equivalent to claiming that treatment and control, while different, follow a similar trajectory
Ideally, you justify by observing the outcome over many pre-treatment periods
What happens if we break parallel trends?
What happens if we break parallel trends?
What happens if we break parallel trends?
Variants of the design
Many groups, treatments, time periods
Increasingly common: Units become treated at different time periods
Example: Policy adopted by cities over a time period
This makes similar to a staggered adoption design
But things get very complicated without randomization
Multiple treatment periods
Example
Leininger et al (2023)
Temporary disenfranchisement in Germany
Discrepancies of minimum voting age across elections
(municipal, state, national)16-17 year olds in Schleswig-Holstein can vote in local but not national elections
Temporary disenfranchisement may push voters away from democracy
Research design
- Outcomes: Survey questions about internal/external efficacy, satisfaction with democracy, political interest
Parallel trends
Results
Thank you!
Break time!
