Causal Inference

Problem 1

Consider a study of the effect of college tutoring programs (X) on final exam scores (Y). Students self-select into tutoring, and prior GPA (C) affects both whether students seek tutoring and their exam performance.

a) Generate data reflecting this structure:

set.seed(1)
n <- 300
prior_gpa <- rnorm(n, mean = 3.0, sd = 0.5)
tutoring <- rbinom(n, 1, plogis(-2 + 1.5 * prior_gpa))
exam_score <- 65 + 8 * tutoring + 10 * prior_gpa + rnorm(n, sd = 5)

Draw the DAG for this scenario. What is the true average treatment effect?

b) Compute and compare two estimates: the naive comparison of means and a regression adjusting for prior GPA. Which is unbiased and why?

c) Explain what “controlling for prior GPA” means. What would happen if prior GPA was actually caused by tutoring instead?

Problem 2

A tech company analyzes employee performance data. They want to understand if coding skills (X) and communication skills (Y) are related. However, they only have data on employees who were hired (M), and hiring depends on both skills.

a) Generate data where coding and communication skills are independent in the full applicant pool:

set.seed(1)
n <- 1000
coding_skill <- rnorm(n, mean = 70, sd = 15)
communication_skill <- rnorm(n, mean = 70, sd = 15)
hire_score <- 0.4 * coding_skill + 0.4 * communication_skill + rnorm(n)
hired <- as.numeric(hire_score > quantile(hire_score, 0.6))

Draw the DAG. What type of variable is hired?

b) Analyze the relationship between coding and communication skills separately for all applicants and for hired employees only. Create two visualizations, one in the whole population and one in just the hired population.

c) Explain the mechanism behind what you observe. What would happen if you ran a regression of communication on coding using only the hired sample?

Problem 3

You want to estimate the effect of online course format (X) on student satisfaction (Y):


Attaching package: 'ggdag'
The following object is masked from 'package:stats':

    filter
library(dagitty)

course_dag <- dagify(
  X ~ instructor_experience + class_size,
  Y ~ X + instructor_experience + engagement,
  engagement ~ class_size,
  exposure = "X",
  outcome = "Y"
)

a) Visualize this DAG and identify all paths from X to Y. Which paths must be blocked to estimate the causal effect?

b) Determine all valid adjustment sets. Is there more than one? Which would you prefer and why?

c) Generate data consistent with this DAG (choose your own coefficients, with true effect = 5). Verify that different valid adjustment sets give similar estimates of the causal effect.

Problem 4

A software company tests whether code review requirements (exposure) improve code quality (outcome). The effect varies by developer experience level:

set.seed(1)
n <- 500
experience_years <- rnorm(n, mean = 5, sd = 3)
experience_years <- pmax(experience_years, 0)
code_review <- rbinom(n, 1, plogis(-1 + 0.2 * experience_years))
code_quality <- 70 + 8 * code_review + 3 * experience_years + 
                0.6 * code_review * experience_years + rnorm(n, sd = 6)

a) Fit an appropriate model and interpret all coefficients. At what experience level is the treatment effect equal to 10?

b) Implement g-computation to estimate the ATE.

c) Why does g-computation give a different answer than the main effect coefficient? Create a visualization showing how the treatment effect varies with experience and how g-computation accounts for the distribution of experience in your sample.

d) What would happen to your ATE estimate if you recruited more junior developers? More senior developers? Explain using your visualization.

Problem 5

A university studies the effect of study group participation (X) on exam scores (Y). They measure high school GPA but fail to measure intrinsic motivation (U).

a) Generate data with an unmeasured confounder:

set.seed(1)
n <- 400
high_school_gpa <- rnorm(n, mean = 3.2, sd = 0.5)
motivation <- rnorm(n, mean = 50, sd = 10)
study_group <- rbinom(n, 1, plogis(-2 + 0.8 * high_school_gpa + 0.04 * motivation))
exam_score <- 50 + 6 * study_group + 8 * high_school_gpa + 
              0.3 * motivation + rnorm(n, sd = 5)

Draw the complete DAG (including the unmeasured confounder).

b) Estimate the effect of study groups three ways: (1) no controls, (2) controlling for high school GPA only, (3) controlling for both (pretend you could measure motivation). Which estimates are biased?

c) Using the omitted variable bias formula, predict the bias in estimate (2). Verify your prediction matches the actual bias.

Problem 6

A research team studies the effect of remote work policy (X) on employee productivity (Y). They have data on:

set.seed(1)
n <- 600
job_autonomy <- rnorm(n, mean = 50, sd = 12)
home_environment <- rnorm(n, mean = 60, sd = 15)
personality_fit <- 0.3 * job_autonomy + 0.2 * home_environment + rnorm(n, sd = 10)
remote_work <- rbinom(n, 1, plogis(-2 + 0.04 * job_autonomy + 0.03 * home_environment + 0.02 * personality_fit))
productivity <- 70 + 5 * remote_work + 0.3 * job_autonomy + 0.2 * home_environment + 0.25 * personality_fit + rnorm(n, sd = 8)

a) Draw the DAG for this data generating process. Identify all confounders of the relationship between remote work and productivity.

b) List all backdoor paths from X to Y. For each path, explain whether it is open or closed by default.

c) Find all valid adjustment sets using adjustmentSets(). Compare estimates using: (1) no controls, (2) each individual valid adjustment set, and (3) all variables. Do the valid adjustment sets give similar answers?

d) Suppose personality_fit was actually unmeasured. Using the omitted variable bias formula, quantify how much bias this would introduce when adjusting only for job_autonomy and home_environment. Would your conclusions about remote work change?