# Need input on which features to drop in classification model

This is the correlation of features with my target variables. I have done all the features engineering but I am left with these features.

Any input on what columns to keep for model training and what to drop. Is there any criteria for dropping features that I don't need. It seems credit history is the only feature that has high correlation.

Loan_ID              0.011610
Gender               0.017987
Married              0.091478
Education           -0.085884
Self_Employed       -0.003700
ApplicantIncome     -0.004710
CoapplicantIncome   -0.059187
LoanAmount          -0.037318
Loan_Amount_Term    -0.022549
Credit_History       0.561678
Total_Income        -0.031271
Total_Income_Log     0.007240
LoanAmt_Log         -0.037536
CH__0               -0.540556
CH__1                0.432616
EMI                 -0.011552
EMI_Log             -0.028496
Dependents_1        -0.038740
Dependents_2         0.062384
Dependents_3        -0.026123
Property_Area_1      0.136540
Property_Area_2     -0.043621
Loan_Status          1.000000
Name: Loan_Status, dtype: float64


Welcome to the community Sai!

Let's assume your problem is a regression problem (i.e. you have continues target).

There are some points:

1. First of all there is no written rule for this. Feature Engineering is an EDA kind of thing. There is no final solution. Your model selection strategy will choose some.
2. Just as a reminder, if these are Pearson correlations, be careful that nonlinear correlations might exist which is not captured by linear correlation analysis. Plus the fact that linear correlation analysis always comes with visual inspection.
3. Negative correlations inhibits information. If whenever a variable increases the other decreases, then knowing it tells you about the other one! So take them into account. You better use Mutual Information for checking dependencies.
4. Sparse Linear Models seem to be fruitful here. I suggest letting LASSO or Ridge Regression choose the final set of features.
5. In case you really insist on your current way (e.g. if your supervisor asked for it), use the threshold as the hyperparameter of your model selection and find the optimal. It means, you train and validate your model using different threshold (which results in different sets of features) and according to empirical error (validation error) you choose the best threshold.

Hope it helped. Good Luck!

• Yes these are Pearson correlations. Does kendall correlation captures nonlinear correlations? – Sai Kumar Oct 25 '18 at 4:59
• Kendall and Spearman (and other rank correlations) are about monotonicity of variations. They capture some nonlinearities but in very specific circumstances. I explained this problem in my course. Still Mutual Information or Maximal Correlation Coefficient or Maximal Information Coefficient etc. – Kasra Manshaei Oct 25 '18 at 15:03