I've recently been reading a lot of papers and watching a lot of videos on both subspace learning, and matrix factorization. One thing is particularly eluding me though - how does any of this get tested?

Let's take, straight from Wikipedia, Matrix Factorization Non-Negative.

$$V = WH$$

So, you have a data matrix $V$. Your goal is to learn components $W$ and $H$, which when multiplied together, give a good approximation of $V$. This can be done by minimizing over $W$, $H$

$$\| V - WH \|$$

That seems fine so far. My problem, theoretically, is understanding when we want to apply this to a problem, like say Regression.

If you wanted to minimize:

$$Y - WH*B$$

How do you do this with a test point? I get confused here, because if we had, say a 100-user test set with 10 features. Then we do a 90/10 split, we get a size of $W*H$ that is different than the size of our test data.

Do people just plug the test data in directly when testing, in place of $W*H$, and just rely on those learned weights $B$?

  • $\begingroup$ I think your analogy is wrong. We don't do any matrix factorization in linear regression. We try to find $W$ in $Y \approx WX$ where $Y$ and $X$ are given. In matrix factorization $Y \approx WH$, we want to find $W$ and $H$ and only $Y$ is given. $\endgroup$
    – Esmailian
    Commented Apr 8, 2019 at 11:12
  • $\begingroup$ Appreciate the reply. Is there a reason, other than the problem above, why we can't do matrix factorization for linear regression? I was trying to think of use-cases where you want to complete a matrix with missing data, but traditional columnwise or rowwise imputation may not make sense. $\endgroup$
    – Jibril
    Commented Apr 8, 2019 at 11:16


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.