1
$\begingroup$

I am doing a prediction of house trade money.
Here is the correlation matrix of features whose correlations are larger than 0.3 as follows:

train_corr_full.unstack().sort_values(kind='quicksort', ascending=False).reset_index()
corr = corr[(corr['level_0'] != corr['level_1'])]
corr

    level_0             level_1              0
13  pv                  uv                   0.911879
14  uv                  pv                   0.911879
15  area                bathroom             0.721935
16  bathroom            area                 0.721935
17  area                tradeMoney           0.687447
18  tradeMoney          area                 0.687447
19  bathroom            tradeMoney           0.580745
20  tradeMoney          bathroom             0.580745
21  bathroom            room                 0.421762
22  room                bathroom             0.421762
23  remainNewNum        totalNewTradeMoney   0.417114
24  totalNewTradeMoney  remainNewNum         0.417114
25  buildYear           totalFloor           0.393571
26  totalFloor          buildYear            0.393571
27  remainNewNum        tradeMeanPrice       0.314611
28  tradeMeanPrice      remainNewNum         0.314611



corr[corr['level_0'] == 'tradeMoney']

    level_0     level_1             0
18  tradeMoney  area                0.687447
20  tradeMoney  bathroom            0.580745
34  tradeMoney  tradeMeanPrice      0.282720
45  tradeMoney  totalFloor          0.249755
47  tradeMoney  tradeNewMeanPrice   0.236713
55  tradeMoney  room                0.215041
79  tradeMoney  buildYear           0.123065
81  tradeMoney  totalTradeMoney     0.122407
95  tradeMoney  remainNewNum        0.100921
116 tradeMoney  pv                  0.072919
134 tradeMoney  uv                  0.040452
137 tradeMoney  totalNewTradeMoney  0.038420

Features descriptions:
area: the area of the house.
room: the number of the room in this house.
bathroom: the number of the bathroom in this house.
totalFloor: total number of floors of this building.
tradeMeanPrice: the mean price of second-hand housing transaction price this month.
tradeNewMeanPrice: the mean price of new housing transaction price this month.
buildYear: the age of this house.
totalTradeMoney: the total price of second-hand housing transaction price this month.
totalNewTradeMoney: the total price of the new housing transaction price this month.
remainNewNum: the number of houses that haven't been sold this month.
pv: the number of times that the website was viewed by tenants.
uv: the number of tenants that viewed the website.

It shows that there is a high correlation with pv and uv but both of which has a low correlation with the target.
What I did before is that I would drop the features whose correlation is lower than about 0.15 and do more EDAs and feature engineerings in those high correlations feature.
And even if consider context I also think the pv and uv is not good enough for the prediction.

Could anyone give some hints on it?
Thanks in advance.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Are you trying to do statistical inference or are you purely looking for improving predictive performance? If it is the latter, I question why you seem to care so much about correlation between features. Correlation can affect the stability of your model (leading to large fitted coefficients and high variance predictions) if you are using a typical linear regression (glm). However, most other algorithms are fairly indifferent to collinearity between variables. Instead of removing variables based off some arbitrary threshold (and therefore, also committing yourself... $\endgroup$
    – aranglol
    Commented May 16, 2019 at 2:22
  • 3
    $\begingroup$ ...to using zero information from a variable), why not try an algorithm that controls for colinearity? Algorithms like ridge regression/partial least squares, support vector machines, tree based learners, etc. $\endgroup$
    – aranglol
    Commented May 16, 2019 at 2:24

1 Answer 1

Reset to default
1
$\begingroup$

If two features have strong correlation together that could mean one of these features is redundant, and probably advised to delete it from the features list in your predictive model.

add to that if the correlation with the target is low then you may need to delete both.

How to decide?

there are many ways, one of them is Feature Selection Filter Method, or an algorithm such as FCBF.

which you try the correlation of the features before implementing your predictive model

The other one is the Wrapper method or Embedded method which will use your predictive model to decide which features to choose.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.