Should I scale my features?

Question

I have a dataset that looks something like this;

ID    | Location | Job_title | blue_jumper | red_jumper | yellow_jumper | green_jumper | Target(purple_jumper)
-------------------------------------------------------------------------------------------------------------------
B7372 | Rome     | Builder   | 2           | 1          | 0             | 9            | 1
D9823 | London   | Lawyer    | 0           | 1          | 8             | 3            | 0
E8718 | Rome     | Teacher   | 3           | 0          | 2             | 3            | 1

etc etc.....

What I would like to do is to use this information to predict whether a person will have a Purple jumper (Target 1 or 0)

Things to note about this dataset I think are the following;

1. I have an ID that relates to the individual
2. I have a number of catergorical features
3. I have some information relating to features that are of the same type (jumpers) but differing by some aspect (colour)
4. These features are of the same type as the Target (e.g jumpers)
5. The target is binary (e.g. I am not looking to predict how many Purple jumpers a person has, just whether they have one or not)

As the Target is binary I know I could use a classification method but I have decided to use Multiple Linear Regression. I like this algorithm because it gives me a measure out that is equivalent to how much like a 1 or 0 my record is.

I have generated dummies for my categorical features but what I am struggling with is whether or not to scale my other features in a situation like this.

Answer 1

It is not necessary to scale your numerical values when using linear regression. However, people still do it since it can speed up training if the algorithm uses gradient descent and it might make your coefficient and intercept terms more interpretable.

You can read more about it here: https://stats.stackexchange.com/questions/185624/feature-scaling-normalization-in-multiple-regression-analysis-with-normal-equa

On another note, there are a lot of binary classifiers that will give you a probability instead of a binary output. For instance, in sklearn many binary classifiers have a predict_proba method that does just this. I can recommend random forest.

Answer 2

Scaling your input variables in Linear Regression has both advantages and disadvantages.

By not scaling you let the parameters, i.e. weights, to have significance by themselves. If the weight(featureA) = 50 it means that an increase of 1 in feature A will increase the target by 50. Nonetheless, the features may have completely different natural ranges (number of bedrooms vs square feet), which may slow the training speed of algorithms based on gradient descend.

If you decide to scale, you may have an increase in speed but you loose the intrinsic interpretability of the parameters. Nonetheless, you can now compare relative feature importances, since they will be forced to a common range of values.

Since your target is a Binary Classification I advise you using the Logistic Regression, which is still a linear algorithm but it ends with a sigmoid function which forces the outputs to be between y ∈ [0, 1].

$$ sigmoid(x) = \frac{\mathrm{1} }{\mathrm{1} + e^{-x} } $$

Similarly to your MLR, Logistic Regression allows for interpretability in the features' weights.