Let us assume we have $k \times k$ matrix valued data and assume this is organized (possibly as time series): $$ M_1, M_2, \ldots, M_n $$

Now, assume we are interested in writing down an error function that mimics sums of squares. This can naively be written as $$ \sum_{i=1}^n (M_i - \hat M_i)^2 $$

where $\hat M_i$ is the $i$-th estimation. The question is, what is actually the proper way to write this function explicitly? For vectors, the Euclidean norm is "naturally" picked. What about this case?

One option is to multiply out these matrices and treat each of the resulting matrix's elements on its own. For example the element at position 11 would have its own "error function" that looks like:

$$ \sum_i (a_{11}^2 +a_{12}a_{21}) $$ and similarly for the other three elements. Here $M-\hat M \equiv A = (a)_{ij}$. Does this even make sense?

Furthermore, how to treat the same example having complex valued matrices?


1 Answer 1


Essentially you want to pick a function that will give you the "size" of a matrix. The most obvious way I can think of is by choosing a matrix norm, which is a map $\lVert \cdot \rVert \colon \mathbb{R}^{k, k} \to [0, \infty)$ (or you could generalise to a complex $k \times k$ matrix if you wished).

Your suggestion seems similar to computing $$S = \sum_i (M_i - \hat M_i)^2$$ then using the Frobenius norm $\lVert S \rVert_F$ to turn this into a real number. The Frobenius norm essentially means "squash $S$ into a vector of dimension $k \times k$, then compute the Euclidean norm".

There are plenty of other norms you could consider, such as the operator norms. Interestingly enough it can be proved that any two matrix norms are equivalent up to a constant scaling factor, so your choice of norm isn't particularly important, and minimising the "sum of squares" under any norm should be more or less the same.

  • $\begingroup$ Indeed. But, as you have seen in my suggestion I am trying to "construct" a matrix valued error as well. So this is why I made this particular choise (i.e. $a_{11}^2 + a_{12}a_{21}$ and so on for the rest of the matrix elements. Thus, I dont want to just compare size of matrices, but construct a new matrix that takes into account correlations between the elements in each $i$. $\endgroup$
    – Marion
    Commented Feb 11, 2021 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.