Evaluating the causal impact of public policies is central to social science research. Whether it’s the introduction of a minimum wage, a new education policy, or a public smoking ban, policymakers and researchers alike want to know: Did the policy make a difference? One of the most widely used empirical strategies to answer such questions is the Difference-in-Differences (DiD) method. This technique is powerful, intuitive, and relatively simple to implement—given that one understands its assumptions and limitations.
In this post, we’ll walk through the fundamentals of DiD, including its key assumption—the parallel trends assumption. We’ll also explain how treatment and control groups are defined and interpreted. Finally, we’ll look at a practical case study evaluating the effect of minimum wage laws using DiD.
What is Difference-in-Differences?
At its core, Difference-in-Differences is a quasi-experimental design that estimates a policy’s causal effect by comparing the changes in outcomes over time between a group that is exposed to a policy (treatment group) and a group that is not (control group).
Unlike simple before-and-after comparisons, DiD accounts for common time trends that may affect both groups. It effectively removes confounding trends that would bias estimates in a purely cross-sectional or purely time-series analysis.
The DiD Estimator
The logic behind DiD is simple. Let’s say we want to evaluate the impact of a new policy (e.g., a new minimum wage law) implemented in State A (treatment group) but not in State B (control group). We observe the average outcome (e.g., employment levels) in both states before and after the policy is implemented. The DiD estimator is:
In words, we subtract the before-after change in the control group from the before-after change in the treatment group. This difference removes time effects common to both groups, allowing us to isolate the effect of the policy.
The Parallel Trends Assumption
The parallel trends assumption is the cornerstone of the DiD method. It states that, in the absence of treatment, the average change in the outcome variable would have been the same across both treatment and control groups.
This is a strong assumption, and it is not directly testable. However, researchers can assess its plausibility using pre-treatment trends—if both groups followed similar trends before the intervention, it strengthens the case for parallel trends.
If this assumption holds, DiD provides an unbiased estimate of the treatment effect. If not, the estimate may reflect differences in trends rather than the causal impact of the policy.
Treatment vs. Control Groups
A proper DiD analysis hinges on correctly identifying:
- Treatment Group – The population that is affected by the policy.
- Control Group – A similar population that is not affected but serves as a benchmark.
The control group acts as a counterfactual—a representation of what would have happened to the treatment group had the policy not been implemented.
Choosing a Control Group
A good control group should be:
- Unaffected by the treatment.
- Similar in observable and unobservable characteristics to the treatment group.
- Following similar trends before the treatment period.
When the control group differs systematically, researchers often use techniques like matching or propensity score weighting to make the groups more comparable.
DiD with Regression
While the basic DiD calculation can be done with group averages, it is more robust to use regression analysis. A simple DiD regression model is:
Where:
- Yit is the outcome for unit i at time t,
- Treatmenti is a binary indicator for being in the treatment group,
- Postt is a binary indicator for the post-treatment period,
- Treatmenti×Postt is the DiD interaction term.
Here, β3 is the DiD estimator—it captures the treatment effect.
Regression allows for:
- Control of covariates, improving precision.
- Clustering standard errors by group.
- Extension to multiple time periods or staggered adoption.
A Case Study: Evaluating the Impact of Minimum Wage Laws
Let’s bring DiD to life with a real-world example. A classic study by Card and Krueger (1994) used a DiD approach to evaluate the effect of a minimum wage increase in New Jersey on employment in the fast-food industry.
The Setup
- Policy change: New Jersey increased its state minimum wage from $4.25 to $5.05 in April 1992.
- Treatment group: Fast-food restaurants in New Jersey.
- Control group: Fast-food restaurants in Eastern Pennsylvania, which did not change its minimum wage.
The researchers collected employment data from both states before and after the policy change.
The Question
Did the higher minimum wage reduce employment in New Jersey’s fast-food restaurants compared to Pennsylvania’s?
Findings
Surprisingly, the study found no evidence of job loss. In fact, employment slightly increased in New Jersey relative to Pennsylvania.
| State | Before (Jobs per restaurant) | After (Jobs per restaurant) | Change |
|---|---|---|---|
| New Jersey | 20.4 | 21.0 | +0.6 |
| Pennsylvania | 21.0 | 20.7 | -0.3 |
| DiD Estimate | +0.9 |
This positive DiD estimate suggested that raising the minimum wage did not reduce employment, challenging the conventional wisdom at the time.
Common Pitfalls and Solutions
Despite its popularity, DiD is prone to several issues:
1. Violating Parallel Trends
- Solution: Check pre-treatment trends using event studies or placebo tests.
2. Differential Composition
- If the composition of the treatment/control group changes over time, the estimate may be biased.
- Solution: Use individual fixed effects or control for changing covariates.
3. Anticipation Effects
- If people expect the policy, they might change behavior before the implementation.
- Solution: Shorten the window around the treatment, or exclude pre-treatment “anticipation” periods.
4. Treatment Spillovers
- If the control group is indirectly affected (e.g., firms in Pennsylvania raise wages to compete), the estimate will be biased.
- Solution: Choose geographically or economically insulated control groups.
Extensions of DiD
Modern research extends DiD in several directions:
1. Staggered DiD
Policies often roll out at different times across units. Researchers now use event-study designs or stacked DiD methods to estimate treatment effects in such contexts.
2. Triple Differences (DDD)
Adds another layer of comparison to control for additional confounders. For example, DiD across industries and regions.
3. Synthetic Control Methods
Construct a weighted combination of control units to better approximate the treated unit.
Conclusion
Difference-in-Differences is a remarkably versatile tool for policy evaluation. It helps researchers go beyond mere correlations and uncover plausible causal effects—provided its assumptions are met. The method’s elegance lies in its simplicity: by comparing the evolution of outcomes across treated and untreated groups, we can isolate the effect of a policy in a real-world setting.
However, practitioners must tread carefully. Ensuring the parallel trends assumption, selecting the right control group, and interpreting results cautiously are vital. With growing extensions and robust estimation tools, DiD remains at the heart of empirical policy analysis.
As a next step, you might consider applying DiD to datasets in education policy, labor markets, or public health. Whether you’re a student, a policymaker, or an early-career researcher, mastering DiD opens up a world of opportunities to evaluate the policies that shape our lives.