How do I get latent features for a new row of data when doing non-negative matrix factorization?

Question

Background: The basic set-up for non-negative matrix factorization (nmf), is that we take a matrix with non-negative elements, X and find two other non-negative matrices H and T such that

$||X - HT||$ is a minimum

$X$ in this example is a matrix that represents a data set. Rows correspond to samples of data, and columns correspond to features. I understand that nmf is used for feature reduction. My understanding is that you can use the latent features to train a model on.

However, suppose that I use nmf for feature reduction and then get a new row of data and want to find the reduced features for the new data. Do I have to run nmf again with an updated dataset and then retrain my model each time?

Answer 1

Given a dataset represented by a data matrix $X$, we can solve $||X-HT||$ using gradient descent, setting coefficients equal to zero if they become negative. This gives us an $H$. If we get a new row of data $x_{new}$, then we can find the corresponding reduced features $w_{new}$(weights for the latent features) by solving the equation:

$argmin_{w_{new}} = ||x_{new}-w_{new}T||$.

Answer 2

@sebastianspiegel is correct and you should accept their answer. That said, there's something to be added in terms of how you might want to practically go about achieving this.

Most NMF libraries give you the option of providing a pre-trained factor/basis matrix, as well as some flag to indicate that you want the algorithm to omit the update step on that matrix. This allows you to take a pre-trained $T$ matrix, and fit a new $H$ matrix to it based on your new data.

There are a couple of nice benefits to this:

1) This factorisation should be much, much quicker to run because your $X$ matrix is now a $m \times 1$ matrix, rather than $m \times n$

2) The features from your previous factorisation should be the same features in the current one, (i.e. if you're factorising movie ratings, and your first factorisation's fifth feature was "scary movies", the feature spat out by this procedure should also be "scary movies")