I am working with PUBG data and developing a linear regression model for the same ! Now there were three features in my original dataset, ridedistance, swimdistance, walkdistance. I combined the three with a new feature : distance covered which is the sum of the above mentioned three features. When putting it in a linearregression model, when I use the three features and the fourth one as well, I get a better score as compared to using the three featues only or using only the fourth feature. I have read that correlation between features when developing a model should not be there. But when all features (4 of them) having correlation are used to develop a model, the model has a better square (R-square). Why is this happening ?
It seems that you are dealing with the problem of Multicollinearity. Multicollinearity happens when your predictors are correlated with other predictors in the model.
Moderate multicollinearity may not be problematic. However, severe multicollinearity is a problem because it can increase the variance of the coefficient estimates and make the estimates very sensitive to minor changes in the model. The result is that the coefficient estimates are unstable and difficult to interpret.
You can use adjusted R-squared to see if the new added variable is actually helping your model to better explain the variance.