Handling categorical variables in linear regression and random forest

Question

In the linear regression, when we have a categorical explanatory variable with $n$ levels, we usually remove one level and call it a baseline level and fit the model on the remaining levels. And the final intercept is the intercept plus the coefficient of baseline level. Now my questions are:

1. Does it matter which level I choose to remove? I am working on a dataset to predict house price. When I use linear model with intercept on the testset for prediction, I get negative values for some houses which is not correct, as house price cannot be negative. But when I fit a regression without an intercept all the prediction values are positive as expected. I guess it has to do something with the baseline level I chose.
2. Do I have to remove one level from all my categorical variables in the dataset before fit a random forest, KNN, Ridge, Lasso,... too?!

Answer 1

1. For linear regression, we have to do one hot encoding and it creates one less number of variables then levels of the categorical variable. In latest tools you don't have to do it manually it automatically does I have tried in R. One hot encoding has no impact on which level you choose, it is basically a binary way of representing the level True or False. This will create whole new variables which with just 1 and 0.

2. For tree-based algorithms, you basically don't need to apply the one hot encoding. These algorithms are capable enough to handle the categorical variables.

Reference: https://gerardnico.com/wiki/data_mining/dummy