Consider a regression with two sets of predictors: \[y = X_1\beta_1 + X_2\beta_2 + \varepsilon\]

Question: What does \(\hat{\beta}_1\) represent?

Answer: The effect of \(X_1\) after controlling for \(X_2\)

Theorem: The OLS estimate \(\hat{\beta}_1\) from the full regression \[y = X_1\beta_1 + X_2\beta_2 + \varepsilon\]

is identical to the OLS estimate from the regression \[\tilde{y} = \tilde{X}_1\beta_1 + \nu\]

• \(\tilde{y}\) are the residuals from regressing \(y\) on \(X_2\)
  
• \(\tilde{X}_1\) are the residuals from regressing \(X_1\) on \(X_2\)

The FWL theorem shows that \(\hat{\beta}_1\) captures the relationship between:

The part of \(y\) that \(X_2\) cannot explain

and

The part of \(X_1\) that \(X_2\) cannot explain

Implication: Multiple regression “partials out” the effects of other variables

Problem: Simple regression of \(y\) on \(x_1\) confounds the effects of \(x_1\) and \(x_2\)

Step 1: Regress \(y\) on \(X_2\) and save residuals \(\tilde{y}\)

Interpretation: \(\tilde{y}\) is the part of \(y\) that \(x_2\) cannot explain

Step 2: Regress \(X_1\) on \(X_2\) and save residuals \(\tilde{X}_1\)

Interpretation: \(\tilde{x}_1\) is the part of \(x_1\) that \(x_2\) cannot explain

Step 3: Regress \(\tilde{y}\) on \(\tilde{X}_1\)

Result: The coefficient equals \(\hat{\beta}_1\) from the full model!

They are identical!

The slope in Step 3 equals \(\hat{\beta}_1\) from the full regression

Definition: \[M_2 = I - X_2(X_2^TX_2)^{-1}X_2^T\]

This is the matrix that creates residuals when you regress something on \(X_2\)

When you regress any variable \(z\) on \(X_2\), the residuals are: \[\text{residuals} = z - X_2(X_2^TX_2)^{-1}X_2^Tz = M_2z\]

Examples: - \(\tilde{y} = M_2 y\) (residuals from regressing \(y\) on \(X_2\)) - \(\tilde{X}_1 = M_2 X_1\) (residuals from regressing \(X_1\) on \(X_2\))

Geometric interpretation: \(M_2\) projects onto the space orthogonal to \(X_2\)

\(M_2\) removes the part of any variable that can be explained by \(X_2\), leaving only the part that is uncorrelated with \(X_2\)

Symmetric: \(M_2^T = M_2\)

Idempotent: \(M_2^2 = M_2\)

Getting residuals from residuals gives the same residuals

Regress \(\tilde{y}\) on \(\tilde{X}_1\): \[\hat{\beta}_1^{FWL} = (\tilde{X}_1^T\tilde{X}_1)^{-1}\tilde{X}_1^T\tilde{y}\]

Substitute \(\tilde{X}_1 = M_2X_1\) and \(\tilde{y} = M_2y\): \[\hat{\beta}_1^{FWL} = [(M_2X_1)^T(M_2X_1)]^{-1}(M_2X_1)^T(M_2y)\]

Expand the transpose: \[\hat{\beta}_1^{FWL} = (X_1^TM_2^TM_2X_1)^{-1}X_1^TM_2^TM_2y\]

Since \(M_2\) is symmetric, \(M_2^T = M_2\): \[\hat{\beta}_1^{FWL} = (X_1^TM_2M_2X_1)^{-1}X_1^TM_2M_2y\]

\[\hat{\beta}_1^{FWL} = (X_1^TM_2M_2X_1)^{-1}X_1^TM_2M_2y\]

Since \(M_2\) is idempotent, \(M_2M_2 = M_2\): \[\hat{\beta}_1^{FWL} = (X_1^TM_2X_1)^{-1}X_1^TM_2y\]

From the partitioned regression formula (normal equations): \[\hat{\beta}_1 = (X_1^TM_2X_1)^{-1}X_1^TM_2y\]

The formulas are identical! \[\hat{\beta}_1^{FWL} = \hat{\beta}_1 \quad \checkmark\]

Both procedures measure the same thing: the relationship between \(X_1\) and \(y\) after accounting for \(X_2\)

FWL approach: Remove \(X_2\)’s influence first, then find relationship

Full regression: Find all relationships simultaneously

\(\hat{\beta}_1\) = effect of \(X_1\) on \(y\) holding \(X_2\) constant

FWL shows this equals: correlation between parts of \(X_1\) and \(y\) that are uncorrelated with \(X_2\)

Common statement: “We control for \(X_2\)”

FWL interpretation: We estimate the effect of the part of \(X_1\) that is orthogonal to \(X_2\)

Example: Effect of education on wages, controlling for experience

• We’re looking at variation in education that is not explained by experience
• And its relationship with the part of wages not explained by experience

Added variable plots (also called partial regression plots) visualize the FWL theorem

Construction:

1. Residualize \(y\) against all controls
2. Residualize the variable of interest against all controls
3. Plot these residuals against each other

Result: The slope of this plot is the coefficient from the full regression

Fixed effects models are a direct application of FWL

Model with individual fixed effects: \[y_{it} = \alpha_i + X_{it}\beta + \varepsilon_{it}\]

FWL approach: Demean everything within each individual \[\tilde{y}_{it} = y_{it} - \bar{y}_i, \quad \tilde{X}_{it} = X_{it} - \bar{X}_i\]

Then regress \(\tilde{y}\) on \(\tilde{X}\) to get \(\hat{\beta}\)

Both methods give the same coefficient!

FWL helps us understand omitted variable bias

Suppose the true model is: \[y = X_1\beta_1 + X_2\beta_2 + \varepsilon\]

But we only estimate: \[y = X_1\beta_1 + u\]

Biased regression estimate: \[\hat{\beta}_1^{\text{bias}} = (X_1^TX_1)^{-1}X_1^Ty\]

Bias: \[E[\hat{\beta}_1^{\text{bias}}] = \beta_1 + (X_1^TX_1)^{-1}X_1^TX_2\beta_2\]

FWL insight: The bias depends on:

1. How \(X_1\) and \(X_2\) are related: \((X_1^TX_1)^{-1}X_1^TX_2\)
2. The true effect of the omitted variable: \(\beta_2\)

Left: Omitting \(x_2\) biases \(\hat{\beta}_1\) because \(x_1\) and \(x_2\) are correlated

Right: No bias because \(x_1\) and \(x_2\) are uncorrelated

Problem: Large number of controls (e.g., many fixed effects)

Solution: Use FWL to avoid inverting huge matrices

Instead of: \[\hat{\beta} = (X^TX)^{-1}X^Ty\] where \(X\) has many columns

Use FWL to focus on the variables of interest after partialing out the controls

Generalization: FWL works when \(X_1\) contains multiple variables

If we want coefficients on several variables after controlling for others: \[y = X_1\beta_1 + X_2\beta_2 + \varepsilon\]

All of \(\hat{\beta}_1\) can be obtained by:

1. Residualize \(y\) on \(X_2\)
2. Residualize each column of \(X_1\) on \(X_2\)
3. Regress residualized \(y\) on residualized \(X_1\)

Key insight: FWL creates orthogonality

After residualizing, \(\tilde{X}_1\) is orthogonal to \(X_2\): \[\tilde{X}_1^TX_2 = (M_2X_1)^TX_2 = X_1^TM_2^TX_2 = 0\]

Implication: In the residualized regression, there’s no confounding between \(\tilde{X}_1\) and \(X_2\)

Important: FWL gives correct coefficient estimates but NOT correct standard errors

Why? The residualized regression uses \(n\) observations but we’ve already “used up” degrees of freedom estimating \(X_2\)’s effects

Solution: Always use the full regression for inference

Use FWL for: Understanding, visualization, and computation

Use full regression for: Hypothesis tests and confidence intervals

Coefficient is identical, but SE differs!