Since one of the columns can be generated completely from the others, and hence retaining this extra column does not add any new information for the modelling process, would it be good practice to always drop the first column after performing One Hot encoding, regardless of the algorithm of choice ?
This question, in a slightly different form, was discussed herein earlier.
You are kind of right, but the best and safest way is to do One-Hot-Encoding and drop at the end because which column you want to drop at the very beginning?
In fact in
pandas.get_dummies there is a parameter i.e.
drop_first allows you whether to keep or remove the reference (whether to keep k or k-1 dummies out of k categorical levels). Please note
default = False meaning that the reference is not dropped and k dummies created out of k categorical levels! You set
default = True, then it will drop the reference column after encoding.