I am working with the kaggle Blueberry Yield prediction dataset. There are 17 columns including the target variable. Below is the correlation heat map:
It can be seen that multiple features are correlated among themselves in a linear manner. Therefore, to find the non-linear relationships, I proceeded with
variance_inflation_factor and this is the result:
What I made out of this VIF table is that:
1. Almost every feature is correlated with one or more other features. Therefore, there exists excessive multicorrelation.
2. If I take the VIF threshold to be 5 and remove the features exceeding this, no feature excluding `honeybee` will be left.
3. Columns 5 to 16 (16th is the target variable) shows abnormally high VIF score!
So, in order to proceed with regression model fitting, if I remove every highly multi-correlated feature, I will be left only with
How do I treat the result of the above VIF score, so that I won't go wrong with the model fitting, which features should I take?