How do you predict a continuous variable when all your independent variables are categorical

Question

I am new to data science and ML. Recently I have been given a sales dataset which contains weekly sales of a fashion brand. It has information about the product like category(t shirt, polo shirt, cotton shirts, briefs, jeans, etc.), gender (male, female, unisex), partner stores ( multiple retail outlets), the color of the apparel (some 150 color codes in hex format), MRP, Sold MRP and quantity sold. The data is for two years timeframe.

I added a new column called discount which is deriver from sold mrp/qty and mrp.

Can anyone suggest any kind of predictive modelling scenario for the above kind of data? I have already done time series forecasting and clustering based on high-performing partner stores. I tried Regression but how do you perform regression when most of the independent variables are categorical in nature ?

Any help would be much appreciated.

Answer 1

A good place to start is with Analysis of Variance (ANOVA) models. The simplest case is where the response/outcome variable is continuous and you have 1 categorical predictor. This is called one-way ANOVA. With 2 categorical predictors you have a 2-way ANOVA and so on. With more than one predictor, interactions between the predictors are also typically included. Of course there is no requirement for all the predictors to be categorical. If one or more are continuous then you would have an Analysis of Covariance (ANCOVA) model. ANOVA and ANCOVA models are just special cases of the General Linear Model (note: not the Generalized Linear Model). In other words it is just linear regression. If data are grouped, for example with repeated measures in subjects, or nested, for example students in classes, or patients in hospitals, then there will be correlations within groups and this can be handled with random effects in a linear mixed model.

Answer 2

In principle, you can do a regression with only factors as explanatory variables. Consider the example (in R):

df = data.frame(c(100,200,500,100,300), c(1,0,1,0,1), c("True", "False", "False", "False", "True"), c("A", "B", "B", "A", "A"))
colnames(df) = c("sales", "v1", "v2", "v3")
head(df)

reg = lm(sales~as.factor(v1)+as.factor(v2)+as.factor(v3), data=df)
summary(reg)

The data looks like:

  sales v1    v2 v3
1   100  1  True  A
2   200  0 False  B
3   500  1 False  B
4   100  0 False  A
5   300  1  True  A

The result will be:

Call:
lm(formula = sales ~ as.factor(v1) + as.factor(v2) + as.factor(v3), 
    data = df)

Residuals:
         1          2          3          4          5 
-1.000e+02  2.842e-14 -2.132e-14 -7.105e-15  1.000e+02 

Coefficients:
                  Estimate Std. Error t value Pr(>|t|)
(Intercept)          100.0      141.4   0.707    0.608
as.factor(v1)1       300.0      200.0   1.500    0.374
as.factor(v2)True   -200.0      264.6  -0.756    0.588
as.factor(v3)B       100.0      200.0   0.500    0.705

Residual standard error: 141.4 on 1 degrees of freedom
Multiple R-squared:  0.8214,    Adjusted R-squared:  0.2857 
F-statistic: 1.533 on 3 and 1 DF,  p-value: 0.5216

So here you measure the difference from the intercept in case $v1=1$ or $v2=True$ or $v3=B$.

Models with a lot (!) of factors have been employed for prediction, but they are often of high dimension (more columns than rows). In this case you could use Lasso to "shrink" parameters (so factor levels where each level is one column) which are not "useful" for prediction.

You can do this when you have a continuous left hand side variable (regression) or as well when you have a discrete variable (Logit, Multinominal Logit).

See Introduction to Statistical Learning (Ch. 6) for more background (including R or Python examples).

Answer 3

It depends on the problem statement, have you got some background about the source of this dataset.. if not then predicting sales in the most common type of analysis you can do here.

And regression doesn't really only depend on input values, it is also considering the past output values so do not worry if the input values are all categorical. You can also try gradient boosting regression trees for predicting sales in partner stores.

Aprt from predictive models, its also a good idea to do some data analysis and find trends and patterns within your data by utilizing graphs

Answer 4

Start with Logistic Regression, NaiveBayes and SVMs. Linear regression does not work well on Categorical data even after encoding. As mentioned, you can encode your categories using Pd.dummy(One hot encoding), Label Encoder. Drop one of the one hot column to follow the assumptions of Logistic Regression. Also Normalize/Standardise your continuous data and also treat your NA's.

Once you are satisfied, then move to DNN's for better accuracy.

I hope this helps.