# non-binary nominal variable in linear regression

For example, I have a variable Color with 3 different values. How can I transform it, so that I can use this variable in linear model? I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables? Shouldn't independent variables be not dependent on each other?

I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables?

Yes, you can use dummy variables to represent a multilevel qualitative variable like Color.

## ISLR Example

An example that may be helpful to you can be found in ISLR, chapter 3 on pages 85 and 86:

The results in the book can be recreated using the Credit data set.

In the example given in the book, balance is regressed onto ethnicity. That is, balance is the quantitative dependent variable (if it were qualitative this would be a classification problem rather than a regression problem), and ethnicity is a 3-level qualitative variable.

> Credit <- read.csv("http://www-bcf.usc.edu/~gareth/ISL/Credit.csv", header = TRUE)
> attach(Credit)
> summary(Balance)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.00   68.75  459.50  520.01  863.00 1999.00

> summary(Ethnicity)
African American            Asian        Caucasian
99              102              199


The task is to predict Balance based on Ethnicity. Therefore Ethnicity needs to be dummified.

As stated in the text pictured above,

There will always be one fewer dummy variable than the number of levels. The level with no dummy variable—African American in this example—is known as the baseline.

is.Caucasian <- ifelse(Ethnicity=="Caucasian", 1, 0)
is.Asian <- ifelse(Ethnicity=="Asian", 1, 0)


The above code accomplishes creating dummy variables for Caucasian and Asian, with African American as the baseline. Now to create the linear model:

balance.lm <- lm(Balance ~ is.Caucasian + is.Asian, Credit)
summary(balance.lm)

Call:
lm(formula = Balance ~ is.Caucasian + is.Asian, data = Credit)

Residuals:
Min      1Q  Median      3Q     Max
-531.00 -457.08  -63.25  339.25 1480.50

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)    531.00      46.32  11.464   <2e-16 ***
is.Caucasian   -12.50      56.68  -0.221    0.826
is.Asian       -18.69      65.02  -0.287    0.774
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 460.9 on 397 degrees of freedom
Multiple R-squared:  0.0002188, Adjusted R-squared:  -0.004818
F-statistic: 0.04344 on 2 and 397 DF,  p-value: 0.9575


The results are an exact match to the information displayed in Table 3.8 on page 86.

## Example using 3 colors

Let's say we wanted to predict the mass of an ball using its color. In this case quantitative variable mass is the response (dependent variable) and color is the independent variable.

> color <- c("red", "black", "green", "red", "green", "red", "red")
> mass <- c(   12,      40,       1,    15,       2,    14,    14)
> ball.data <- data.frame(mass, color)

mass color
1   12   red
2   40 black
3    1 green
4   15   red
5    2 green
6   14   red


We can dummify color into two variables, one for red and one for black, with green as the baseline.

> ball.data$is.red <- ifelse(ball.data$color=="red", 1, 0)
> ball.data$is.black <- ifelse(ball.data$color=="black", 1, 0)

mass color is.red is.black
1   12   red      1        0
2   40 black      0        1
3    1 green      0        0
4   15   red      1        0
5    2 green      0        0
6   14   red      1        0


Then we fit a linear model like so:

> ball.lm <- lm(mass ~ is.red + is.black, ball.data)
> summary(ball.lm)

Call:
lm(formula = mass ~ is.red + is.black, data = ball.data)

Residuals:
1          2          3          4          5          6          7
-1.750e+00  5.551e-17 -5.000e-01  1.250e+00  5.000e-01  2.500e-01  2.500e-01

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)   1.5000     0.8101   1.852 0.137727
is.red       12.2500     0.9922  12.347 0.000247 ***
is.black     38.5000     1.4031  27.439 1.05e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.146 on 4 degrees of freedom
Multiple R-squared:  0.9947,    Adjusted R-squared:  0.9921
F-statistic: 376.7 on 2 and 4 DF,  p-value: 2.79e-05


When we compare the coefficient estimates in the linear model summary to the values in the data set, we see they are pretty close, as expected. That is about it.