The dummy variable trap is a common problem with linear regression when dealing with categorical variables, since one hot encoding introduces redundancy, so if we have m categories in our categorical variable we usually drop one dummy variable to have m-1 dummy variables instead of m dummy variables. Now, when talking about linear regression we must do so. The reason behind this can be quoted from the book Data Preparation for Machine Learning: Data Cleaning, Feature Selection, and Data Transforms in Python by Jason Brownlee , page 245 :
For example, in the case of a linear regression model (and other regression models that have a bias term), a one hot encoding will cause the matrix of input data to become singular, meaning it cannot be inverted and the linear regression coefficients cannot be calculated using linear algebra. For these types of models a dummy variable encoding must be used instead.
So according to these words, if we use one hot encoding without removing one dummy variable, we will not be able to get the coefficients of the model, however, in many cases we can still use one hot encoding with linearregression()
class provided by sklearn library and the coefficients are then found easily by this class. So my first question is So my first question is cancan someone explainsexplain the statement mention bymentioned in this book because hereit seems to me we have a contradiction probably because I misunderstood something.
Moreover, when using drop="first"drop="first"
with onehotencoding()
, we cant set the handle_unknown
attribute to "ignore"
, so in the case when we have unknown category in the test set or in any prediction we will get an error, because we don't have any options rather than to set the handle_unknown
attribute to "error"
. After doing a research to fix such problem, many people suggest to use one hot encodingencoding; so this brings me to another question, so this brings me to another question, howhow can we use one hot encoding if this encoding method introduces the dummy variable trap ?