Skip to main content
1 of 2
Peter
  • 7.8k
  • 5
  • 22
  • 50

In principle, you can do a regression with only factors as explanatory variables. Consider the example:

df = data.frame(c(100,200,500,100,300), c(1,0,1,0,1), c("True", "False", "False", "False", "True"), c("A", "B", "B", "A", "A"))
colnames(df) = c("sales", "v1", "v2", "v3")
head(df)

reg = lm(sales~as.factor(v1)+as.factor(v2)+as.factor(v3), data=df)
summary(reg)

The data looks like:

  sales v1    v2 v3
1   100  1  True  A
2   200  0 False  B
3   500  1 False  B
4   100  0 False  A
5   300  1  True  A

The result will be:

Call:
lm(formula = sales ~ as.factor(v1) + as.factor(v2) + as.factor(v3), 
    data = df)

Residuals:
         1          2          3          4          5 
-1.000e+02  2.842e-14 -2.132e-14 -7.105e-15  1.000e+02 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
(Intercept)          100.0      141.4   0.707    0.608
as.factor(v1)1       300.0      200.0   1.500    0.374
as.factor(v2)True   -200.0      264.6  -0.756    0.588
as.factor(v3)B       100.0      200.0   0.500    0.705

Residual standard error: 141.4 on 1 degrees of freedom
Multiple R-squared:  0.8214,    Adjusted R-squared:  0.2857 
F-statistic: 1.533 on 3 and 1 DF,  p-value: 0.5216

So here you measure the difference from the intercept in case $v1=1$ or $v2=True$ or $v3=B$.

Models with a lot (!) of factors have been employed for prediction, but they are often of high dimension. In this case you could use Lasso to "shrink" parameters (so factor levels) which are not "useful" for prediction.

You can do this when you have a continuous left hand side variable (regression) or as well when you have a discrete variable (Logit, Multinominal Logit).

See Introduction to Statistical Learning (Ch. 6) for more background (including R or Python examples).

Peter
  • 7.8k
  • 5
  • 22
  • 50