After optimizing the MSE (mean squared error) in a regression task, how is the change in Pearson correlation coeficient between target vector and the prediction?
Is any behaviour possible? Or is sure that it becomes larger or lowers?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
It should not be lower but it does not always have to be higher.
Let's consider these two vectors:
A | B 1 | 2 2 | 3 3 | 4
The correlation between A & B is super high as I've just modeled a deterministic relationship (B = A +1). However if I perceive B as a predicted value and A as the real value then the MSE would not be 0.
Now let's say I improve my prediction and it's perfect:
A | B 1 | 1 2 | 2 3 | 3
We haven't impacted the correlation between those vectors at all because we simply removed the constant shift between the vectors. Their relation is still 1:1, however the MSE if B is the prediction and A is the real value is now much, much better than in the first example.