I wonder about the excessive usage of the covariance matrix across all kinds of machine learning tools. So far, for me, the covariance is just a pre-step to get to the correlation. And as there is an obvious reason for the correlation itself, I wonder why I encounter the covariance so often. And, however, I wonder in general why it is used so much. What is/are the purposes for the covariance matrix?
Is essential when u look at theory of linear models, matrix algebra and also u can see usefulness in Methods of multivariate analysis book (because in ML u do use more than 1 variables variables so is explained in details theory behind that).
In simple words: covariance matrix show the
distribution magnitude and measure of directionalrelationship between variables
for multivariate data in multidimensional space and useful tool for
decorrelate variables or applied as transformation for other variables. Here is math-info cor and cov cor vs cov in ML space
Also, is very useful for models and special for big data modeling, dimension reductions aka PCA and family. I use it for feature reduction too and there u can use from cov matrix or corr matrix to extract requirements and build PCs and more (this is a-lot to explain here and also is part of explaining PCA).
Hope it helps and can add math explanation if is the case.