Since one of the columns can be generated completely from the others, and hence retaining this extra column does not add any new information for the modelling process, would it be good practice to always drop the first column after performing One Hot encoding, regardless of the algorithm of choice ?
This question, in a slightly different form, was discussed herein earlier.
You are kind of right, but the best and safest way is to do One-Hot-Encoding and drop at the end because which column you want to drop at the very beginning?
In fact in
pandas.get_dummies there is a parameter i.e.
drop_first allows you whether to keep or remove the reference (whether to keep k or k-1 dummies out of k categorical levels). Please note
drop_first = False meaning that the reference is not dropped and k dummies created out of k categorical levels! You set
drop_first = True, then it will drop the reference column after encoding.
This has something to do with Mutli-colinearity in case if Multiple Linear Regression. Beacause, Keeping k dummies for k levels of a categorical variable is good idea, but there is a redundancy of one level, which is here in separate column. This is not needed since one of the combination will be uniquely representing this redundant column. Hence, its better to drop one of the column and just have k-1 dummies(columns) to represent k levels.
This Overall approach reduces Multi-colinearity in the dataset, which is one of the prime Assumption of Multiple Linear Regression.