I am working with the kaggle Blueberry Yield prediction dataset. There are 17 columns including the target variable. Below is the correlation heat map:


It can be seen that multiple features are correlated among themselves in a linear manner. Therefore, to find the non-linear relationships, I proceeded with statsmodels's variance_inflation_factor and this is the result:


What I made out of this VIF table is that:

1. Almost every feature is correlated with one or more other features. Therefore, there exists excessive multicorrelation.

2. If I take the VIF threshold to be 5 and remove the features exceeding this, no feature excluding `honeybee` will be left.

3. Columns 5 to 16 (16th is the target variable) shows abnormally high VIF score!

So, in order to proceed with regression model fitting, if I remove every highly multi-correlated feature, I will be left only with honeybee and yield.

How do I treat the result of the above VIF score, so that I won't go wrong with the model fitting, which features should I take?


1 Answer 1


Suggestion for features taking:

  • Techniques like backward elimination, forward selection, and recursive feature elimination can help you select the most important features.
  • highly correlated features can be combined by adding or multiplications with other features.
  • Then you can use PCA(Principal Component Analysis) to reduce the dimensionality because your features will be increased then it will be the curse of dimensionality.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.