When do we scale features and should it be done to label encoded features?

Question

So, I am trying to make a linear regression model for predicting car prices for which I have the following data set: Data Set

Since the unique values are a lot for the categorical features, I label encoded the categorical features and converted the data set like this: Modified Data Set

Thereafter, I scaled the continuous features ( Engine, Power) like so:

# Scaling continuous features (No scaling for categorical variables)

X_train[['Engine', 'Power']] = scale.fit_transform(X_train[['Engine', 'Power']])


est = sm.OLS(y_train,X_train).fit()
print(est.summary())

X_test[['Engine', 'Power']] = scale.transform(X_test[['Engine', 'Power']])
y_predicted = est.predict(X_test)
y_predicted.sort_values()

# R2 value for y_test and y_predictions

from sklearn.metrics import r2_score 

r2 = r2_score(y_test, y_predicted) 
print(r2)

and got an R2 value of 84.8% compared to 33.5% if I also scaled label encoded features. I want to know why this happened? Also, my predicted values using this model on testing data is resulting in negative prices as well and most of these prices are not predicted well with the R2 score of predicted and true data being 68%.

Answer 1

You should not use Label Encoding for Categorical data unless there is a known ranking and that also in the specified ratio between the level values.
In this case, the model will assume 10 as 2 times of 5.

One-hot will add a lot of dimensions as I can see in your data.

You must try other Categorical encoding techniques esp. Sum Coding Or Helmert.

You should also try Dimension reduction techniques post-One-Hot to see if it reduces the dimensions.

Please do read these references -

Beyond-one-hot
Library
Kaggle tutorial
Feature Engg