0
$\begingroup$

For example, I have a variable Color with 3 different values. How can I transform it, so that I can use this variable in linear model? I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables? Shouldn't independent variables be not dependent on each other?

$\endgroup$
2
$\begingroup$

I've read about dummy variables, but as these three will depend on each other (if the value is "red", then it's not "black"), can I use dummy variables?

Yes, you can use dummy variables to represent a multilevel qualitative variable like Color.

ISLR Example

An example that may be helpful to you can be found in ISLR, chapter 3 on pages 85 and 86:

ISLR page 85 ISLR page 86

The results in the book can be recreated using the Credit data set.

In the example given in the book, balance is regressed onto ethnicity. That is, balance is the quantitative dependent variable (if it were qualitative this would be a classification problem rather than a regression problem), and ethnicity is a 3-level qualitative variable.

> Credit <- read.csv("http://www-bcf.usc.edu/~gareth/ISL/Credit.csv", header = TRUE)
> attach(Credit)
> summary(Balance)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00   68.75  459.50  520.01  863.00 1999.00 

> summary(Ethnicity)
African American            Asian        Caucasian 
              99              102              199 

The task is to predict Balance based on Ethnicity. Therefore Ethnicity needs to be dummified.

As stated in the text pictured above,

There will always be one fewer dummy variable than the number of levels. The level with no dummy variable—African American in this example—is known as the baseline.

is.Caucasian <- ifelse(Ethnicity=="Caucasian", 1, 0)
is.Asian <- ifelse(Ethnicity=="Asian", 1, 0)

The above code accomplishes creating dummy variables for Caucasian and Asian, with African American as the baseline. Now to create the linear model:

balance.lm <- lm(Balance ~ is.Caucasian + is.Asian, Credit)
summary(balance.lm)

Call:
lm(formula = Balance ~ is.Caucasian + is.Asian, data = Credit)

Residuals:
    Min      1Q  Median      3Q     Max 
-531.00 -457.08  -63.25  339.25 1480.50 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)    531.00      46.32  11.464   <2e-16 ***
is.Caucasian   -12.50      56.68  -0.221    0.826    
is.Asian       -18.69      65.02  -0.287    0.774    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 460.9 on 397 degrees of freedom
Multiple R-squared:  0.0002188, Adjusted R-squared:  -0.004818 
F-statistic: 0.04344 on 2 and 397 DF,  p-value: 0.9575

The results are an exact match to the information displayed in Table 3.8 on page 86.

Example using 3 colors

Let's say we wanted to predict the mass of an ball using its color. In this case quantitative variable mass is the response (dependent variable) and color is the independent variable.

> color <- c("red", "black", "green", "red", "green", "red", "red")
> mass <- c(   12,      40,       1,    15,       2,    14,    14)
> ball.data <- data.frame(mass, color)
> head(ball.data)

  mass color
1   12   red
2   40 black
3    1 green
4   15   red
5    2 green
6   14   red

We can dummify color into two variables, one for red and one for black, with green as the baseline.

> ball.data$is.red <- ifelse(ball.data$color=="red", 1, 0)
> ball.data$is.black <- ifelse(ball.data$color=="black", 1, 0)
> head(ball.data)

  mass color is.red is.black
1   12   red      1        0
2   40 black      0        1
3    1 green      0        0
4   15   red      1        0
5    2 green      0        0
6   14   red      1        0

Then we fit a linear model like so:

> ball.lm <- lm(mass ~ is.red + is.black, ball.data)
> summary(ball.lm)

Call:
lm(formula = mass ~ is.red + is.black, data = ball.data)

Residuals:
         1          2          3          4          5          6          7 
-1.750e+00  5.551e-17 -5.000e-01  1.250e+00  5.000e-01  2.500e-01  2.500e-01 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   1.5000     0.8101   1.852 0.137727    
is.red       12.2500     0.9922  12.347 0.000247 ***
is.black     38.5000     1.4031  27.439 1.05e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.146 on 4 degrees of freedom
Multiple R-squared:  0.9947,    Adjusted R-squared:  0.9921 
F-statistic: 376.7 on 2 and 4 DF,  p-value: 2.79e-05

When we compare the coefficient estimates in the linear model summary to the values in the data set, we see they are pretty close, as expected. That is about it.

More info

PSU STAT 501: Regression Methods, Lesson 8: Categorical Predictors

CODING FOR CATEGORICAL VARIABLES IN REGRESSION MODELS | R LEARNING MODULES

| improve this answer | |
$\endgroup$
0
$\begingroup$

When two variables are independent of each other, it means that no variable can be expressed as a function of the other. "If value is red, then it's not black" isn't an example of dependent variables.

I am assuming that your '3 variables' are R,G and B in the range (0-255). To transform the 3 variables into a singular one for a linear model, you could use hex valued colors. Look at this one to get a better idea: https://math.stackexchange.com/questions/1635999/algorithm-to-convert-integer-to-3-variables-rgb

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.