1
$\begingroup$

I am working on a real state data set to predict the price of buying a house in Dubai based on area, no.of bedrooms, number of baths and the town which the house is in.

All variables are numerical except the town variable which I transformed, using one hot encoding and target transform.

Then I applied linear regression, decision tree, random forest and I got the same results for both transforming methods.

Algorithm            Train score        Test score  

linear regression    0.50               0.45
decision tree        0.93               0.79
random forest        0.94               0.77

From the results, I can read(but not so sure) that the data has information and has a high correlation scores but the model is over fitting. I used grid search to optimize the hyper parameters of the decision tree but the result did not improve.

So, the question is, what am I doing wrong?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Here are some questions that might help:

  1. Do you have a balanced dataset? How does the distribution look like?
  2. Is the distribution of the training and test set similar?
  3. Did you try using cross validation?
  4. What is your accuracy metric?
  5. Are the features correlated among themselves?
  6. I would also suggest using xgboost and compare its results.
Improve this answer
$\endgroup$
8
  • $\begingroup$ 3. I used cross_validate from sklearn and got a score around 0.5 but in sklearn there is no way to apply cross_validate on decision tree and pick a final model from it.(as i understood from multiple articles) $\endgroup$
    – Mohammad Nur
    Commented Nov 6, 2019 at 8:27
  • $\begingroup$ 4.The accuracy metric I used is R^2 $\endgroup$
    – Mohammad Nur
    Commented Nov 6, 2019 at 8:27
  • $\begingroup$ 5.The correlation for all features is between 0.5-0.9 except for the variable that resulted from applying target transform to the county variable which its correlation is around 0.1 $\endgroup$
    – Mohammad Nur
    Commented Nov 6, 2019 at 8:31
  • $\begingroup$ I was trying to suggest something like K Fold cross validation, you can take the mean of all the models rather than taking one. I would suggest using the mean absolute error (MAE) and RMSE to evaluate your results, that would allow you to identify which examples are not doing well on the model. How many country names do you have? Target transform might not be a good approach I would suggest looking at generating dummy variables for regression. Also you should try to visualize the country features and see if they make any sense as a predictor or just drop them. $\endgroup$
    – Samarth
    Commented Nov 6, 2019 at 16:27
  • $\begingroup$ There are more than 100 different county names and they have some information because prices change significantly between different counties and when i excluded them the training result went down. I tried dummy variables but got the same results. $\endgroup$
    – Mohammad Nur
    Commented Nov 6, 2019 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.