weekdays <- c("Monday","Tuesday","Wednesday",
"Thursday","Friday","Saturday",
"Sunday")
class( weekdays )[1] "character"6 Factor Data
Factors allow us to represent a type of data that exclusive and categorical. These data may, or may not, be ordered and in most cases, we can think of these kinds of data to represent things like, treatment levels, sampling locations, etc.
Lecture Materials
Below are the slides that are used in the lecture component of this topic. You can cycle through them using the cursor arrows or open the online version in a new browser window.
Factors
I’m going to start with some days of the week because they are exclusive (e.g., you cannot be in both Monday and Wednesday at the same time). Factors are initially created from string objects, though you could use numeric data but that would be stupid because you should use things that are descriptive and strings are much better than that.
I’m going to take these days and random sample them to create a vector of 40 elements. This is something we do all the time and there is a sample() function that allows us to draw random samples either with or without replacement (e.g., can you select the same value more than once).
data <- sample( weekdays, size=40, replace=TRUE)
data [1] "Monday" "Friday" "Monday" "Tuesday" "Wednesday" "Sunday"
[7] "Saturday" "Wednesday" "Monday" "Monday" "Friday" "Thursday"
[13] "Thursday" "Monday" "Wednesday" "Monday" "Saturday" "Wednesday"
[19] "Monday" "Tuesday" "Tuesday" "Tuesday" "Saturday" "Monday"
[25] "Sunday" "Thursday" "Tuesday" "Friday" "Monday" "Tuesday"
[31] "Friday" "Monday" "Friday" "Thursday" "Thursday" "Thursday"
[37] "Friday" "Wednesday" "Tuesday" "Wednesday"These data are still
class( data )[1] "character"To turn them into a factor, we use…. factor()
days <- factor( data )
is.factor( days )[1] TRUEclass( days )[1] "factor"Now when we look at the data, it looks a lot like it did before except for the last line which shows you the unique levels for elements in the vector.
days [1] Monday Friday Monday Tuesday Wednesday Sunday Saturday
[8] Wednesday Monday Monday Friday Thursday Thursday Monday
[15] Wednesday Monday Saturday Wednesday Monday Tuesday Tuesday
[22] Tuesday Saturday Monday Sunday Thursday Tuesday Friday
[29] Monday Tuesday Friday Monday Friday Thursday Thursday
[36] Thursday Friday Wednesday Tuesday Wednesday
Levels: Friday Monday Saturday Sunday Thursday Tuesday Wednesdaysummary( days ) Friday Monday Saturday Sunday Thursday Tuesday Wednesday
6 10 3 2 6 7 6 We can put them into data frames and they know how to summarize themselves properly by counting the number of occurances of each level.
df <- data.frame( ID = 1:40, Weekdays = days )
summary( df ) ID Weekdays
Min. : 1.00 Friday : 6
1st Qu.:10.75 Monday :10
Median :20.50 Saturday : 3
Mean :20.50 Sunday : 2
3rd Qu.:30.25 Thursday : 6
Max. :40.00 Tuesday : 7
Wednesday: 6 And we can directly access the unique levels
levels( days )[1] "Friday" "Monday" "Saturday" "Sunday" "Thursday" "Tuesday"
[7] "Wednesday"So factors can be categorical (e.g., one is just different than the next) and compared via == and != values. Or they can be ordinal such that > and < make sense.
By default, a factor is not ordered.
is.ordered( days )[1] FALSEdays[1] < days[2]Warning in Ops.factor(days[1], days[2]): '<' not meaningful for factors[1] NAdata <- factor( days, ordered=TRUE )
data [1] Monday Friday Monday Tuesday Wednesday Sunday Saturday
[8] Wednesday Monday Monday Friday Thursday Thursday Monday
[15] Wednesday Monday Saturday Wednesday Monday Tuesday Tuesday
[22] Tuesday Saturday Monday Sunday Thursday Tuesday Friday
[29] Monday Tuesday Friday Monday Friday Thursday Thursday
[36] Thursday Friday Wednesday Tuesday Wednesday
7 Levels: Friday < Monday < Saturday < Sunday < Thursday < ... < WednesdaySo that if we go and try to order them, the only way they can be sorted is alphabetically.
sort( data ) [1] Friday Friday Friday Friday Friday Friday Monday
[8] Monday Monday Monday Monday Monday Monday Monday
[15] Monday Monday Saturday Saturday Saturday Sunday Sunday
[22] Thursday Thursday Thursday Thursday Thursday Thursday Tuesday
[29] Tuesday Tuesday Tuesday Tuesday Tuesday Tuesday Wednesday
[36] Wednesday Wednesday Wednesday Wednesday Wednesday
7 Levels: Friday < Monday < Saturday < Sunday < Thursday < ... < WednesdayHowever, this does not make sense. Who in their right mind would like to have Friday followed immediately by Monday? That is just not right!
To establish an ordinal variable with a specified sequence of values that are not alphabetical we need to pass along the levels themselves.
data <- factor( days, ordered=TRUE, levels = weekdays )
data [1] Monday Friday Monday Tuesday Wednesday Sunday Saturday
[8] Wednesday Monday Monday Friday Thursday Thursday Monday
[15] Wednesday Monday Saturday Wednesday Monday Tuesday Tuesday
[22] Tuesday Saturday Monday Sunday Thursday Tuesday Friday
[29] Monday Tuesday Friday Monday Friday Thursday Thursday
[36] Thursday Friday Wednesday Tuesday Wednesday
7 Levels: Monday < Tuesday < Wednesday < Thursday < Friday < ... < SundayNow they’ll sort properly.
sort( data ) [1] Monday Monday Monday Monday Monday Monday Monday
[8] Monday Monday Monday Tuesday Tuesday Tuesday Tuesday
[15] Tuesday Tuesday Tuesday Wednesday Wednesday Wednesday Wednesday
[22] Wednesday Wednesday Thursday Thursday Thursday Thursday Thursday
[29] Thursday Friday Friday Friday Friday Friday Friday
[36] Saturday Saturday Saturday Sunday Sunday
7 Levels: Monday < Tuesday < Wednesday < Thursday < Friday < ... < SundayExclusivity of Factor Levels
Once you establish a factor, you cannot set the values to anyting that is outside of the pre-defined levels. If you do, it will just put in missing data NA.
days[3] <- "Bob"Warning in `[<-.factor`(`*tmp*`, 3, value = "Bob"): invalid factor level, NA
generateddays [1] Monday Friday <NA> Tuesday Wednesday Sunday Saturday
[8] Wednesday Monday Monday Friday Thursday Thursday Monday
[15] Wednesday Monday Saturday Wednesday Monday Tuesday Tuesday
[22] Tuesday Saturday Monday Sunday Thursday Tuesday Friday
[29] Monday Tuesday Friday Monday Friday Thursday Thursday
[36] Thursday Friday Wednesday Tuesday Wednesday
Levels: Friday Monday Saturday Sunday Thursday Tuesday WednesdayThat being said, we can have more levels in the factor than observed in the data. Here is an example of just grabbing the work days from the week but making the levels equal to all the potential weekdays.
workdays <- sample( weekdays[1:5], size=40, replace = TRUE )
workdays <- factor( workdays, ordered=TRUE, levels = weekdays )And when we summarize it, we see that while it is possible that days may be named Saturday and Sunday, they are not recoreded in the data we have for workdays.
summary( workdays ) Monday Tuesday Wednesday Thursday Friday Saturday Sunday
6 12 7 6 9 0 0 We can drop the levels that have no representation
workdays <- droplevels( workdays )
summary( workdays ) Monday Tuesday Wednesday Thursday Friday
6 12 7 6 9 The forcats Library
The forcats library has a bunch of helper functions for working with factors. This is a relatively small library in tidyverse but a powerful one. I would recommend looking at the cheatsheet for it to get a more broad understanding of what functions in this library can do.
library( forcats )Just like stringr had the str_ prefix, all the functions here have the fct_ prefix. Here are some examples.
Counting how many of each factor
fct_count( data )# A tibble: 7 × 2
f n
<ord> <int>
1 Monday 10
2 Tuesday 7
3 Wednesday 6
4 Thursday 6
5 Friday 6
6 Saturday 3
7 Sunday 2Lumping Rare Factors
lumped <- fct_lump_min( data, min = 5 )
fct_count( lumped )# A tibble: 6 × 2
f n
<ord> <int>
1 Monday 10
2 Tuesday 7
3 Wednesday 6
4 Thursday 6
5 Friday 6
6 Other 5Reordering Factor Levels by Frequency
freq <- fct_infreq( data )
levels( freq )[1] "Monday" "Tuesday" "Wednesday" "Thursday" "Friday" "Saturday"
[7] "Sunday" Reordering by Order of Appearance
ordered <- fct_inorder( data )
levels( ordered )[1] "Monday" "Friday" "Tuesday" "Wednesday" "Sunday" "Saturday"
[7] "Thursday" Reordering Specific Levels
newWeek <- fct_relevel( data, "Sunday")
levels( newWeek )[1] "Sunday" "Monday" "Tuesday" "Wednesday" "Thursday" "Friday"
[7] "Saturday" Dropping Unobserved Levels - just like droplevels()
dropped <- fct_drop( workdays )
summary( dropped ) Monday Tuesday Wednesday Thursday Friday
6 12 7 6 9 Using Factors
It is common to use factors as an organizing princple in our data. For example, let’s say we went out and sampled three different species of plants and measured characteristics of their flower size. The iris data set from R.A. Fisher is a classid data set that is include in R and it looks like this (the functions head() and tail() show the top or bottom parts of a data frame).
head(iris) Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosaBy default it is a data.frame object.
class( iris )[1] "data.frame"By the by
One helpful function in base R is the by() function. It has the following form.
by( data, index, function)The data is the raw data you are using, the index is a vector that we are using to differentiate among the species (the factor), and the function is what function we want to use.
So for example, if I were interesed in the mean length of the Sepal for each species, I could write.
meanSepalLength <- by( iris$Sepal.Length, iris$Species, mean )
class( meanSepalLength )[1] "by"meanSepalLengthiris$Species: setosa
[1] 5.006
------------------------------------------------------------
iris$Species: versicolor
[1] 5.936
------------------------------------------------------------
iris$Species: virginica
[1] 6.588I could also do the same thing with the variance in sepal length.
by( iris[,2], iris[,5], var ) -> varSepalLength
varSepalLength iris[, 5]: setosa
[1] 0.1436898
------------------------------------------------------------
iris[, 5]: versicolor
[1] 0.09846939
------------------------------------------------------------
iris[, 5]: virginica
[1] 0.1040041Using these kinds of functions we can create a summary data frame.
library( tidyverse )── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr 1.2.1 ✔ readr 2.2.0
✔ ggplot2 4.0.2 ✔ stringr 1.6.0
✔ lubridate 1.9.5 ✔ tibble 3.3.1
✔ purrr 1.2.1 ✔ tidyr 1.3.2
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag() masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errorsdf <- tibble( Species = levels( iris$Species),
Average = meanSepalLength,
Variance = varSepalLength
)
df# A tibble: 3 × 3
Species Average Variance
<chr> <by[1d]> <by[1d]>
1 setosa 5.006 0.14368980
2 versicolor 5.936 0.09846939
3 virginica 6.588 0.10400408Missing Data
Missing data is a .red[fact of life] and R is very opinionated about how it handles missing values. In general, missing data is encoded as NA and is a valid entry for any data type (character, numeric, logical, factor, etc.). Where this becomes tricky is when we are doing operations on data that has missing values. R could take two routes:
- It could ignore the data and give you the answer directly as if the data were not missing, or
- It could let you know that there is missing data and make you do something about it.
Fortunately, R took the second route.
An example from the iris data, I’m going to add some missing data to it.
missingIris <- iris[, 4:5]
missingIris$Petal.Width[ c(2,6,12) ] <- NA
summary( missingIris ) Petal.Width Species
Min. :0.100 setosa :50
1st Qu.:0.300 versicolor:50
Median :1.300 virginica :50
Mean :1.218
3rd Qu.:1.800
Max. :2.500
NA's :3 Notice how the missing data is denoted in the summary.
Indications of Missing Data
When we perform a mathematical or statistical operation on data that has missing elements R will always return NA as the result.
mean( missingIris$Petal.Width )[1] NAThis warns you that .red[at least one] of the observations in the data is missing.
Same output for using by(), it will put NA into each level that has at least one missing value.
by( missingIris$Petal.Width, missingIris$Species, mean )missingIris$Species: setosa
[1] NA
------------------------------------------------------------
missingIris$Species: versicolor
[1] 1.326
------------------------------------------------------------
missingIris$Species: virginica
[1] 2.026Working with Missing Data
To acknowledge that there are missing data and you still want the values, you need to tell the function you are using that data is missing and you are OK with that using the optional argument na.rm=TRUE (na = missing data & rm is remove).
mean( missingIris$Petal.Width, na.rm=TRUE)[1] 1.218367To pass this to the by() function, we add the optional argument na.rm=TRUE and by() passes it along to the mean function as “…”
by( missingIris$Petal.Width, missingIris$Species, mean, na.rm=TRUE )missingIris$Species: setosa
[1] 0.2446809
------------------------------------------------------------
missingIris$Species: versicolor
[1] 1.326
------------------------------------------------------------
missingIris$Species: virginica
[1] 2.026Fancy Tables
Making data frames like that above is a classic maneuver in R and I’m going to use this to introduce the use of the knitr library to show you how to take a set of data and turn it into a table for your manuscript.
library( knitr )Now we can make a table as:
kable( df )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
We can even add a caption to it.
irisTable <- kable( df, caption = "The mean and variance in measured sepal length (in cm) for three species of Iris.")
irisTable| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
In addition to this basic library, there is an kableExtra one that allows us to get even more fancy. You must go check out this webpage (which is an RMarkdown page by the way) to see all the other ways you can fancy up your tables.
library( kableExtra )
Attaching package: 'kableExtra'The following object is masked from 'package:dplyr':
group_rowsTable Themes
Here are some examples Themes
kable_paper( irisTable )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
kable_classic( irisTable )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
kable_classic_2( irisTable )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
kable_minimal( irisTable )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
kable_material( irisTable,lightable_options = c("striped", "hover") )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
kable_material_dark( irisTable )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
Table Sizes and Positions
We can be specific about the size and location of the whole table.
kable_paper(irisTable, full_width = FALSE )| Species | Average | Variance |
|---|---|---|
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |
Heading Judo
We can do some really cool stuff on row and column headings. Here is an example where I add another row above the data columns for output.
classic <- kable_paper( irisTable )
add_header_above( classic, c(" " = 1, "Sepal Length (cm)" = 2))
Sepal Length (cm)
|
||
|---|---|---|
| Species | Average | Variance |
| setosa | 5.006 | 0.14368980 |
| versicolor | 5.936 | 0.09846939 |
| virginica | 6.588 | 0.10400408 |