UCI COSMOS 2026 Noah Benjamin (adapted material from Zhaoxia Yu and Mine Dogucu)
Linear Regression in R
Now that we have learned about regression models, we’ll build a multiple regression model for predicting the left hippocampus volume of the brain, labeled as lhippo, through two predictors, namely age and educ.
Remember that in the general, a multiple linear regression model with \(p\) explanatory variables can be presented as:
The left hand side of this model is the response variable, a numerical continuous variable.
Why would we investigate this relationship?
Recall the left hippocampus volume lhippo is likely to shrink as Alzheimer’s severs.
Also, while the progress of the disease is a function of age, it is possible that education could have a reverse effect on the progress of the disease.
To fit linear models all we need to do is to apply the lm() command in R.
First, though, we will begin by plotting the response versus each predictor, separately.
Plotting response vs. predictor
(left) lhippo vs. age , lhippo vs. educ (right)
Now let’s do some linear regression!
lhippo vs. age
Here is the regression of lhippo versus age:
lm_model <-lm(lhippo ~ age, data = alzheimer_data)summary(lm_model)
Call:
lm(formula = lhippo ~ age, data = alzheimer_data)
Residuals:
Min 1Q Median 3Q Max
-2.58855 -0.28598 0.01999 0.31504 1.58641
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.0639626 0.0543632 74.76 <2e-16 ***
age -0.0149051 0.0007657 -19.46 <2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.4593 on 2698 degrees of freedom
Multiple R-squared: 0.1231, Adjusted R-squared: 0.1228
F-statistic: 378.9 on 1 and 2698 DF, p-value: < 2.2e-16
ggplot(data = alzheimer_data, aes(x = educ, y = lhippo)) +geom_point(color ="red") +geom_smooth(method ="lm", color ="blue", se=FALSE) +labs(x ="Education", y ="Left Hippocampus Volume")
Regressing on both variables
Here is the regression of lhippo versus age and education:
lm_model <-lm(lhippo ~ age + educ, data = alzheimer_data)summary(lm_model)
Call:
lm(formula = lhippo ~ age + educ, data = alzheimer_data)
Residuals:
Min 1Q Median 3Q Max
-2.59525 -0.28746 0.01681 0.31416 1.54719
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 3.7348265 0.0709453 52.644 < 2e-16 ***
age -0.0142527 0.0007643 -18.649 < 2e-16 ***
educ 0.0185428 0.0026010 7.129 1.29e-12 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 0.4551 on 2697 degrees of freedom
Multiple R-squared: 0.1394, Adjusted R-squared: 0.1387
F-statistic: 218.4 on 2 and 2697 DF, p-value: < 2.2e-16
Regressing on both variables
lm(lhippo ~ age + educ, data = alzheimer_data) %>%tbl_regression(estimate_fun =function(x) style_number(x, digits =3))
Characteristic
Beta
95% CI
p-value
age
-0.014
-0.016, -0.013
<0.001
educ
0.019
0.013, 0.024
<0.001
Abbreviation: CI = Confidence Interval
Cross-Validation
Cross-Validation
Let’s try to evaluate the performance of our model by calculating its accuracy or mean squared error (MSE), depending on whether we are dealing with a linear regression or logistic regression model.
Cross-validation is a commonly used technique to achieve this evaluation. It is an old approach, devised by statisticians Fred Mosteller and John Tukey in 1968.
The process involves splitting the data into training and validation (or test) sets. We then fit or train the model using the training portion of the dataset. The accuracy or MSE is then calculated by comparing the predictions made by the model on the validation set to the actual values.
Metrics
For linear regression models, we typically use the mean squared error (MSE) to measure the quality of predictions. The lower the MSE, the better the model’s performance. The MSE represents the average squared difference between the predicted values and the true values.
For classification problems (e.g., logistic regression), we use accuracy as a metric to assess the model’s performance. Accuracy represents the proportion of correctly classified instances out of the total instances in the validation set. The higher the accuracy, the better the model’s performance.
Splitting the data
To split the data into training and validation sets using the rsample package in R, you can use the initial_split() function. Here’s an example of how you can split the data:
The intercept in multiple linear regression model is the expected (average) value of the response variable when all the explanatory variables in the model are set to zero simultaneously.
In the above example, the intercept is \(a = 2791\), which is obtained by setting age and smoking to zero.
We might be tempted to interpret this as the average birthweight of babies for nonsmoking mothers (smoke=0) with age equal to zero. In this case, however, this is not a reasonable interpretation since mother’s age cannot be zero.
We interpret \(b_j\) as our estimate of the expected (average) change in the response variable associated with a unit increase in the corresponding explanatory variable \(x_j\) while all other explanatory variables in the model remain fixed.
For the above example, the point estimate of the regression coefficient for age is \(b_1 = 11\), and the estimate of the regression coefficient for smoke is \(b_2 = −278\).
We expect that the birthweight of babies increase by 11 grams as the mother’s age increases by one year among mothers with the same smoking status.
The expected birthweight changes by \(−278\) (decreases by 278) grams associated with one unit increase in the value of the variable smoke (i.e., going from non-smoking mothers to smoking mothers) among mothers with the same age.
Let’s talk a little about assumptions
For the model to work we assume that these assumptions are met
