Introduction

Previously, we discussed regression models involving a single categorical and single quantitative explanatory variable:

These fitted models look like separate, parallel regression lines for each category of the categorical variable.

Multiple Slopes

When considering models with multiple quantitative variables, let’s start by looking at two separate simple linear regressions:

• In regression #1, the fitted slope is -42814, so each 0.1 increase in admission rate corresponds with an expected decrease in average faculty salary of $4281.40
• In regression #2, the fitted slope is 4276, so each 1-pt increase in median ACT predicts an expected increase of $4276.00

Regression Planes

Below is the regression plane of the model: Avg_Fac_Salary ~ Adm_Rate + ACT_median

• The fitted slopes are -10374 for Adm_Rate and 4032.80 for ACT_median
  • Why do these slopes differ from the ones on the previous slide?

Regression Planes

Recall that the coefficients in a fitted regression model optimize the squared distance between the model’s predictions and the observed values of the response variable:

This is still the case when there are multiple explanatory variables, but if the explanatory variables are correlated the best fitting plane (multivariable model) will not be equivalent to the best fitting lines in each individual variable (separate simple linear regression models):

Adjusted Effects

Median ACT scores are negatively correlated with their admissions rates:

• Simple linear regression provides a marginal effect which ignores this relationship
  • A 1-unit increase in median ACT is associated with a decrease in admissions rate, and the effect of this decrease is “absorbed” into the marginal effect of median ACT on average faculty salary
• Multivariable linear regression adjusts for this relationship by finding coefficients that take into account the simultaneous changes in median ACT and admissions rate that happen in the data
  • We now get an adjusted effect that reflects the separate contributions of each variable when the associated changes in the other variable accounted for

Conclusion

• Simple linear regression estimates marginal effects, while multivariable regression offers us a tool to obtain adjusted effects
  • Multivariable regression finds the best-fitting regression plane, which is not equivalent to individually estimating each explanatory variable’s impact using simple linear regression
• If categorical variables are included, there will be multiple, parallel fitted regression planes with each having a different y-intercept