How to proceed when correlation plot doesn't show as much multicorrelation as is seen by statsmodels variance_inflation_factor in a regression task?

Question

I am working with the kaggle Blueberry Yield prediction dataset. There are 17 columns including the target variable. Below is the correlation heat map:

It can be seen that multiple features are correlated among themselves in a linear manner. Therefore, to find the non-linear relationships, I proceeded with statsmodels's variance_inflation_factor and this is the result:

What I made out of this VIF table is that:

1. Almost every feature is correlated with one or more other features. Therefore, there exists excessive multicorrelation.

2. If I take the VIF threshold to be 5 and remove the features exceeding this, no feature excluding `honeybee` will be left.

3. Columns 5 to 16 (16th is the target variable) shows abnormally high VIF score!

So, in order to proceed with regression model fitting, if I remove every highly multi-correlated feature, I will be left only with honeybee and yield.

How do I treat the result of the above VIF score, so that I won't go wrong with the model fitting, which features should I take?

Answer 1

Suggestion for features taking:

• Techniques like backward elimination, forward selection, and recursive feature elimination can help you select the most important features.
• highly correlated features can be combined by adding or multiplications with other features.
• Then you can use PCA(Principal Component Analysis) to reduce the dimensionality because your features will be increased then it will be the curse of dimensionality.