Frisch-Waugh-Lovell Theorem

Problem 1

Consider the following small dataset:

\[\mathbf{y} = \begin{bmatrix}2\\5\\8\\11\\14\end{bmatrix}, \quad \mathbf{X}_1 = \begin{bmatrix}1\\4\\3\\4\\5\end{bmatrix}, \quad \mathbf{X}_2 = \begin{bmatrix}1\\1\\2\\2\\3\end{bmatrix}\]

a) Fit the full regression model \(y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \varepsilon\) and report \(\hat{\beta}_1\).

b) Now apply the FWL procedure: - Regress \(y\) on \(x_2\) (including an intercept) and save the residuals \(\tilde{y}\) - Regress \(x_1\) on \(x_2\) (including an intercept) and save the residuals \(\tilde{x}_1\) - Regress \(\tilde{y}\) on \(\tilde{x}_1\) (without an intercept)

Report the coefficient from this final regression. Does it match \(\hat{\beta}_1\) from part (a)?

Problem 2

Generate data with three predictors:

set.seed(1)
n <- 100
x1 <- rnorm(n)
x2 <- rnorm(n)
x3 <- 0.6*x1 + 0.4*x2 + rnorm(n)
y <- 3 + 2*x1 + 1.5*x2 + 2.5*x3 + rnorm(n)

a) Fit the full model lm(y ~ x1 + x2 + x3) and report the coefficient on x3.

b) Create an added variable plot for x3: - Regress y on x1 and x2, save residuals - Regress x3 on x1 and x2, save residuals - Plot these residuals against each other and add a regression line

c) What is the slope of the line in your added variable plot? How does it compare to the coefficient from part (a)?

d) Explain what this plot shows: what variation in y and x3 are we visualizing?

Problem 3

For a simple case with \(\mathbf{X}_2 = \begin{bmatrix}1\\1\\1\end{bmatrix}\) (just an intercept):

a) Write out the \(3 \times 3\) residual maker matrix \(\mathbf{M}_2 = \mathbf{I} - \mathbf{X}_2(\mathbf{X}_2^T\mathbf{X}_2)^{-1}\mathbf{X}_2^T\).

b) Verify that \(\mathbf{M}_2\) is symmetric: \(\mathbf{M}_2^T = \mathbf{M}_2\).

c) Verify that \(\mathbf{M}_2\) is idempotent: \(\mathbf{M}_2^2 = \mathbf{M}_2\).

d) Apply \(\mathbf{M}_2\) to the vector \(\mathbf{z} = \begin{bmatrix}5\\8\\11\end{bmatrix}\). What operation does this perform? (Hint: what is the mean of the resulting vector?)

Problem 4

Generate data where an important variable is omitted:

set.seed(1)
n <- 200
x1 <- rnorm(n)
x2 <- 0.8*x1 + rnorm(n) 
y <- 2 + 3*x1 + 4*x2 + rnorm(n)

a) Fit the “short” regression lm(y ~ x1) that omits x2. Report \(\hat{\beta}_1^{\text{short}}\).

b) Fit the “full” regression lm(y ~ x1 + x2). Report \(\hat{\beta}_1^{\text{full}}\).

c) Calculate the auxiliary regression coefficient: fit lm(x2 ~ x1) and report the slope \(\hat{\delta}\).

d) Verify the omitted variable bias formula: \[\hat{\beta}_1^{\text{short}} \approx \hat{\beta}_1^{\text{full}} + \hat{\delta} \cdot \hat{\beta}_2^{\text{full}}\]

Explain intuitively why the bias has this form using FWL reasoning.

Problem 5

Consider panel data with individuals observed over time:

set.seed(1)
id <- rep(1:10, each = 5)
time <- rep(1:5, times = 10)
alpha_i <- rep(rnorm(10, mean = 5), each = 5)
x <- rnorm(50, mean = 10)
y <- alpha_i + 2.5*x + rnorm(50)

a) Fit a model with individual fixed effects using dummy variables:

model_fe <- lm(y ~ x + factor(id))

Report the coefficient on x.

b) Now use the FWL/demeaning approach: - Calculate within-individual means: \(\bar{y}_i\) and \(\bar{x}_i\) for each individual - Create demeaned variables: \(\tilde{y}_{it} = y_{it} - \bar{y}_i\) and \(\tilde{x}_{it} = x_{it} - \bar{x}_i\) - Regress the demeaned outcome on the demeaned predictor (no intercept)

Does this coefficient match part (a)?

c) Explain why demeaning is equivalent to including fixed effects, using the FWL theorem.

Problem 6

Using the data from Problem 1:

a) Compare the standard errors for \(\hat{\beta}_1\) from: - The full regression lm(y ~ x1 + x2) - The FWL residualized regression lm(y_resid ~ x1_resid)

Why do they differ?

b) How many degrees of freedom are used in each regression? Explain how this affects the standard error calculation.