Introduction to data wrangling

Author

Dr. Mine Dogucu

1 Import Data

Code

library(tidyverse)
options(scipen = 999)

arthritis <- read_csv("https://raw.githubusercontent.com/cosmos-uci-dshs/data/main/RheumArth_Tx_AgeComparisons.csv") |>
  janitor::clean_names() |>
  mutate(sex = case_when(sex == 0 ~ "female",
                         sex == 1 ~ "male"),
         sex = as.factor(sex),
         age_gp = case_when(age_gp == 1 ~ "control",
                            age_gp == 2 ~ "elderly"),
         age_gp = as.factor(age_gp),
         cdai_yn = case_when(cdai_yn == 1 ~ "no",
                             cdai_yn == 2 ~ "yes"),
         cdai_yn = as.factor(cdai_yn)) |>
  select(age, age_gp, sex, yrs_from_dx, cdai)

1.1 Data Overview

We are working with the arthritis dataset. Here is a quick look at its structure:

glimpse(arthritis)

Rows: 530
Columns: 5
$ age         <dbl> 85, 86, 83, 83, 85, 79, 90, 90, 87, 82, 77, 86, 84, 76, 77…
$ age_gp      <fct> elderly, elderly, elderly, elderly, elderly, elderly, elde…
$ sex         <fct> female, female, female, female, female, male, female, fema…
$ yrs_from_dx <dbl> 27, 27, 10, 9, NA, NA, 51, 11, 36, 4, 31, NA, 9, 10, 3, 10…
$ cdai        <dbl> NA, 23.0, 14.5, NA, NA, NA, NA, 40.0, 6.0, NA, 0.0, NA, NA…

2 Mutating existing variables

We use mutate() to create or modify variables in a data frame.

2.1 Creating a numeric variable: `age_months`

Goal: Create a new variable called age_months that represents age in months.

Code

arthritis |>
  mutate(age_months = age * 12)

# A tibble: 530 × 6
     age age_gp  sex    yrs_from_dx  cdai age_months
   <dbl> <fct>   <fct>        <dbl> <dbl>      <dbl>
 1    85 elderly female          27  NA         1020
 2    86 elderly female          27  23         1032
 3    83 elderly female          10  14.5        996
 4    83 elderly female           9  NA          996
 5    85 elderly female          NA  NA         1020
 6    79 elderly male            NA  NA          948
 7    90 elderly female          51  NA         1080
 8    90 elderly female          11  40         1080
 9    87 elderly female          36   6         1044
10    82 elderly female           4  NA          984
# ℹ 520 more rows

Note that this does not change the original arthritis object. To save the result, assign it:

Code

arthritis_age_mon <- arthritis |>
  mutate(age_months = age * 12)

glimpse(arthritis_age_mon)

Rows: 530
Columns: 6
$ age         <dbl> 85, 86, 83, 83, 85, 79, 90, 90, 87, 82, 77, 86, 84, 76, 77…
$ age_gp      <fct> elderly, elderly, elderly, elderly, elderly, elderly, elde…
$ sex         <fct> female, female, female, female, female, male, female, fema…
$ yrs_from_dx <dbl> 27, 27, 10, 9, NA, NA, 51, 11, 36, 4, 31, NA, 9, 10, 3, 10…
$ cdai        <dbl> NA, 23.0, 14.5, NA, NA, NA, NA, 40.0, 6.0, NA, 0.0, NA, NA…
$ age_months  <dbl> 1020, 1032, 996, 996, 1020, 948, 1080, 1080, 1044, 984, 92…

2.2 Using `arrange` to sort numeric variable (age)

Code

# aesending order
arthritis |>
  arrange(age)

# A tibble: 530 × 5
     age age_gp  sex    yrs_from_dx  cdai
   <dbl> <fct>   <fct>        <dbl> <dbl>
 1    42 control female           8    20
 2    42 control female           9    12
 3    42 control female          27    18
 4    42 control female           1    NA
 5    42 control female          16    NA
 6    42 control male             2    14
 7    43 control female           2     0
 8    43 control female           7    NA
 9    44 control female           3    NA
10    44 control female          11    NA
# ℹ 520 more rows

Quest: how about sorting age in a descending order?

Code

# descending order
arthritis |>
  arrange(desc(age))

# A tibble: 530 × 5
     age age_gp  sex    yrs_from_dx  cdai
   <dbl> <fct>   <fct>        <dbl> <dbl>
 1    90 elderly female          51    NA
 2    90 elderly female          11    40
 3    90 elderly female          NA    NA
 4    90 elderly female          11    NA
 5    90 elderly female           3    NA
 6    90 elderly female          40    NA
 7    90 elderly female          40    14
 8    88 elderly female          11    NA
 9    88 elderly female          12    NA
10    87 elderly female          36     6
# ℹ 520 more rows

2.3 Creating a categorical variable: `cdai_level`

Goal: Create a new variable called cdai_level based on the CDAI data dictionary:

Level	Condition
Remission	cdai ≤ 2.8
Low Disease Activity	cdai > 2.8 and ≤ 10
Moderate Disease Activity	cdai > 10 and ≤ 22
High Disease Activity	cdai > 22

We use case_when() inside mutate() to assign categories:

Code

arthritis |>
  mutate(cdai_level = case_when(
    cdai <= 2.8             ~ "Remission",
    cdai > 2.8 & cdai <= 10 ~ "Low",
    cdai > 10  & cdai <= 22 ~ "Moderate",
    cdai > 22               ~ "High"))

# A tibble: 530 × 6
     age age_gp  sex    yrs_from_dx  cdai cdai_level
   <dbl> <fct>   <fct>        <dbl> <dbl> <chr>     
 1    85 elderly female          27  NA   <NA>      
 2    86 elderly female          27  23   High      
 3    83 elderly female          10  14.5 Moderate  
 4    83 elderly female           9  NA   <NA>      
 5    85 elderly female          NA  NA   <NA>      
 6    79 elderly male            NA  NA   <NA>      
 7    90 elderly female          51  NA   <NA>      
 8    90 elderly female          11  40   High      
 9    87 elderly female          36   6   Low       
10    82 elderly female           4  NA   <NA>      
# ℹ 520 more rows

To save the result back into arthritis:

Code

arthritis <- arthritis |>
  mutate(cdai_level = case_when(
    cdai <= 2.8             ~ "Remission",
    cdai > 2.8 & cdai <= 10 ~ "Low",
    cdai > 10  & cdai <= 22 ~ "Moderate",
    cdai > 22               ~ "High"))

glimpse(arthritis)

Rows: 530
Columns: 6
$ age         <dbl> 85, 86, 83, 83, 85, 79, 90, 90, 87, 82, 77, 86, 84, 76, 77…
$ age_gp      <fct> elderly, elderly, elderly, elderly, elderly, elderly, elde…
$ sex         <fct> female, female, female, female, female, male, female, fema…
$ yrs_from_dx <dbl> 27, 27, 10, 9, NA, NA, 51, 11, 36, 4, 31, NA, 9, 10, 3, 10…
$ cdai        <dbl> NA, 23.0, 14.5, NA, NA, NA, NA, 40.0, 6.0, NA, 0.0, NA, NA…
$ cdai_level  <chr> NA, "High", "Moderate", NA, NA, NA, NA, "High", "Low", NA,…

2.4 Converting variable types

Goal:

Make cdai_level a factor and age an integer.
Manually define the order of cdai_level to “High”, “Moderate”, “Low”, “Remission”.

Code

arthritis |>
  mutate(
    cdai_level = factor(cdai_level,levels = c("High", "Moderate", "Low", "Remission")),
    age = as.integer(age))

# A tibble: 530 × 6
     age age_gp  sex    yrs_from_dx  cdai cdai_level
   <int> <fct>   <fct>        <dbl> <dbl> <fct>     
 1    85 elderly female          27  NA   <NA>      
 2    86 elderly female          27  23   High      
 3    83 elderly female          10  14.5 Moderate  
 4    83 elderly female           9  NA   <NA>      
 5    85 elderly female          NA  NA   <NA>      
 6    79 elderly male            NA  NA   <NA>      
 7    90 elderly female          51  NA   <NA>      
 8    90 elderly female          11  40   High      
 9    87 elderly female          36   6   Low       
10    82 elderly female           4  NA   <NA>      
# ℹ 520 more rows

2.4.1 Use `arrange()` sorts factors vs. characters:

Code

# Without levels: alphabetical (High, Low, Moderate, Remission)
arthritis |>
  arrange(cdai_level)

# A tibble: 530 × 6
     age age_gp  sex    yrs_from_dx  cdai cdai_level
   <dbl> <fct>   <fct>        <dbl> <dbl> <chr>     
 1    86 elderly female          27  23   High      
 2    90 elderly female          11  40   High      
 3    76 elderly female           7  35   High      
 4    49 control female          10  31   High      
 5    47 control female          37  23   High      
 6    64 control female           9  39   High      
 7    62 control female          30  28   High      
 8    55 control female           8  39.5 High      
 9    54 control female           3  24   High      
10    53 control female           4  28   High      
# ℹ 520 more rows

Code

# With levels: manually defined order
arthritis |>
  mutate(cdai_level = factor(cdai_level,
    levels = c("Remission", "Low", "Moderate", "High"))) |>
  arrange(cdai_level)

# A tibble: 530 × 6
     age age_gp  sex    yrs_from_dx  cdai cdai_level
   <dbl> <fct>   <fct>        <dbl> <dbl> <fct>     
 1    77 elderly female          31     0 Remission 
 2    76 elderly female          10     2 Remission 
 3    77 elderly female          20     2 Remission 
 4    67 control female           8     0 Remission 
 5    55 control female          10     0 Remission 
 6    59 control female           3     0 Remission 
 7    56 control female           9     0 Remission 
 8    70 control female           5     0 Remission 
 9    57 control female           4     0 Remission 
10    51 control female           4     0 Remission 
# ℹ 520 more rows

Common type-conversion functions:

Function	Result
`as.factor()`	factor
`as.numeric()`	numeric
`as.integer()`	integer
`as.double()`	double
`as.character()`	character

2.5 Your Turn

Using the arthritis dataset, complete the two tasks below.

2.5.1 Task 1: Categorize years since diagnosis

Create a new variable called yrs_category that groups yrs_from_dx into three stages:

Category	Condition
Early	yrs_from_dx ≤ 5
Established	yrs_from_dx > 5 and ≤ 15
Long-standing	yrs_from_dx > 15

Then make it a factor with levels ordered from earliest to longest: "Early", "Established", "Long-standing".

Code

arthritis |>
  mutate(
    yrs_category = case_when(
      yrs_from_dx <= 5                        ~ "Early",
      yrs_from_dx > 5  & yrs_from_dx <= 15   ~ "Established",
      yrs_from_dx > 15                        ~ "Long-standing"),
    yrs_category = factor(yrs_category, levels = c("Early", "Established", "Long-standing")))

# A tibble: 530 × 7
     age age_gp  sex    yrs_from_dx  cdai cdai_level yrs_category 
   <dbl> <fct>   <fct>        <dbl> <dbl> <chr>      <fct>        
 1    85 elderly female          27  NA   <NA>       Long-standing
 2    86 elderly female          27  23   High       Long-standing
 3    83 elderly female          10  14.5 Moderate   Established  
 4    83 elderly female           9  NA   <NA>       Established  
 5    85 elderly female          NA  NA   <NA>       <NA>         
 6    79 elderly male            NA  NA   <NA>       <NA>         
 7    90 elderly female          51  NA   <NA>       Long-standing
 8    90 elderly female          11  40   High       Established  
 9    87 elderly female          36   6   Low        Long-standing
10    82 elderly female           4  NA   <NA>       Early        
# ℹ 520 more rows

2.5.2 Task 2: Sort by disease stage and age

Extend your pipeline from Task 1: sort the result first by yrs_category (earliest to longest), then by age in descending order within each stage.

Code

arthritis |>
  mutate(
    yrs_category = case_when(
      yrs_from_dx <= 5                        ~ "Early",
      yrs_from_dx > 5  & yrs_from_dx <= 15    ~ "Established",
      yrs_from_dx > 15                        ~ "Long-standing"),
    yrs_category = factor(yrs_category, levels = c("Early", "Established", "Long-standing"))) |>
  arrange(yrs_category, desc(age))

# A tibble: 530 × 7
     age age_gp  sex    yrs_from_dx  cdai cdai_level yrs_category
   <dbl> <fct>   <fct>        <dbl> <dbl> <chr>      <fct>       
 1    90 elderly female           3    NA <NA>       Early       
 2    87 elderly female           2    NA <NA>       Early       
 3    86 elderly female           4    NA <NA>       Early       
 4    84 elderly female           2    NA <NA>       Early       
 5    83 elderly male             2    NA <NA>       Early       
 6    82 elderly female           4    NA <NA>       Early       
 7    82 elderly male             1    NA <NA>       Early       
 8    82 elderly female           1    NA <NA>       Early       
 9    81 elderly female           3    NA <NA>       Early       
10    81 elderly male             4    NA <NA>       Early       
# ℹ 520 more rows

1 Import Data

1.1 Data Overview

2 Mutating existing variables

2.1 Creating a numeric variable: age_months

2.2 Using arrange to sort numeric variable (age)

2.3 Creating a categorical variable: cdai_level

2.4 Converting variable types

2.4.1 Use arrange() sorts factors vs. characters:

2.5 Your Turn

2.5.1 Task 1: Categorize years since diagnosis

2.5.2 Task 2: Sort by disease stage and age

2.1 Creating a numeric variable: `age_months`

2.2 Using `arrange` to sort numeric variable (age)

2.3 Creating a categorical variable: `cdai_level`

2.4.1 Use `arrange()` sorts factors vs. characters: