Practice Problems

Work through each problem by hand, then use the R code to check your answers.

Problem 1

A multiple linear regression model is fit with \(n = 35\), \(\text{SSE} = 180\), \(\hat{\boldsymbol{\beta}} = (2.0,\; -1.0,\; 4.0)^T\), and \((\mathbf{X}^T\mathbf{X})^{-1} = \begin{bmatrix}0.3 & 0 & 0\\0 & 0.5 & 0\\0 & 0 & 0.2\end{bmatrix}\). Compute the standard errors for all three coefficients. Compute the \(t\)-statistic for each slope and determine which (if any) you reject at \(\alpha = 0.05\). Construct 95% confidence intervals for \(\beta_1\) and \(\beta_2\). For a new observation \(\mathbf{x}_\text{new} = (1, 2, 1)^T\), compute the point prediction, a 95% CI for \(E[y_\text{new}]\), and a 95% PI for \(y_\text{new}\).

n <- 35
p <- 3
SSE <- 180
beta_hat <- c(2.0, -1.0, 4.0)
XtXinv <- diag(c(0.3, 0.5, 0.2))
x_new <- c(1, 2, 1)

sigma2 <- SSE / (n - p)
se <- sqrt(sigma2 * diag(XtXinv))
tcrit <- qt(0.975, df = n - p)

sigma2
[1] 5.625
se
[1] 1.299038 1.677051 1.060660
t <- beta_hat / se
t
[1]  1.5396007 -0.5962848  3.7712362
abs(t) > tcrit
[1] FALSE FALSE  TRUE
beta_hat[2] + c(-1,1) * tcrit * se[2] 
[1] -4.416041  2.416041
beta_hat[3] + c(-1,1) * tcrit * se[3] 
[1] 1.839506 6.160494
yhat <- sum(beta_hat * x_new)
qf <- as.numeric(t(x_new) %*% XtXinv %*% x_new)
yhat
[1] 4
yhat + c(-1,1) * tcrit * sqrt(sigma2 * qf)
[1] -3.6385 11.6385
yhat + c(-1,1) * tcrit * sqrt(sigma2 * (1 + qf))
[1] -5.037995 13.037995

Problem 2

A model with 5 parameters is fit to \(n = 50\) observations, \(\text{SST} = 2000\), and SSE = 800. Compute \(R^2\) and adjusted \(R^2\).

n <- 50
SST <- 2000
p <- 5
SSE <- 800

R2 <- 1 - SSE / SST
R2_adj <- 1 - (SSE / (n - p)) / (SST / (n - 1))

Problem 3

A simple linear regression (\(p = 2\)) is fit on \(n = 7\) observations. The residuals and leverage values are given below.

\(i\) \(e_i\) \(h_{ii}\)
1 \(1.5\) \(0.4\)
2 \(-2.0\) \(0.3\)
3 \(0.5\) \(0.1\)
4 \(0.0\) \(0.1\)
5 \(-1.0\) \(0.2\)
6 \(2.0\) \(0.4\)
7 \(-1.0\) \(0.5\)

Compute the LOOCV error using the shortcut formula. Compute the standardized residuals and flag any with \(|r_i| > 2\).

e <- c(1.5, -2.0, 0.5, 0.0, -1.0, 2.0, -1.0)
h <- c(0.4,  0.3, 0.1, 0.1,  0.2, 0.4,  0.5)
n <- 7
p <- 2
mean((e / (1 - h))^2)
[1] 4.485074
sigma_hat <- sqrt(sum(e^2) / (n - p))
r <- e / (sigma_hat * sqrt(1 - h))

Problem 4

A cubic spline with interior knots at \(\xi_1 = 2\) and \(\xi_2 = 6\) is fit to observations \(x = (1, 4, 6, 9, 11)\) with estimated coefficients \(\hat{\boldsymbol{\beta}} = (0.5,\; 1.2,\; -0.1,\; 0.01,\; -0.02,\; 0.005)^T\). Construct the full design matrix. Compute \(\hat{y}\) for each observation. Compute the expected change in \(\hat{y}\) going from \(x = 4\) to \(x = 9\).

x <- c(1, 4, 6, 9, 11)
beta <- c(0.5, 1.2, -0.1, 0.01, -0.02, 0.005)
tp <- function(x, xi) ifelse(x > xi, (x - xi)^3, 0)

X <- cbind(1, x, x^2, x^3, tp(x, 2), tp(x, 6))


y_hat <- X %*% beta

x_comp <- c(4, 9)
X_comp <- cbind(1, x_comp, x_comp^2, x_comp^3, tp(x_comp, 2), tp(x_comp, 6))
preds <- X_comp %*% beta