I am working on a real state data set to predict the price of buying a house in Dubai based on area, no.of bedrooms, number of baths and the town which the house is in.
All variables are numerical except the town variable which I transformed, using one hot encoding and target transform.
Then I applied linear regression, decision tree, random forest and I got the same results for both transforming methods.
Algorithm Train score Test score
linear regression 0.50 0.45
decision tree 0.93 0.79
random forest 0.94 0.77
From the results, I can read(but not so sure) that the data has information and has a high correlation scores but the model is over fitting. I used grid search to optimize the hyper parameters of the decision tree but the result did not improve.
So, the question is, what am I doing wrong?