How exactly does matrix factorization help with collaborative filtering

Question

We start with a matrix of user ratings for different movies with some elements unknow i.e the rating of a yet to be seen movie by an user. We need to fill in this gap. So

• How can you decompose or factorize a matrix where all elements are not known in advance.
• after you somehow decompose it, do you just multiply the matrices back to recreate the original one, but now with the missing item populated?
• How do you know which factorization method like non negative,singular value,eigen to choose,without going into too much math?

Answer 1

Answers to your question

1. You factorize the matrix in order to approximate original one as closely as possible. This is generally done by starting with randomized values, and updating based on error (between product of factors and original matrix). In other words, for a given matrix A, you are trying to find matrices C & D such that Error(A - (C x D)) is lowest. The algorithm is designed to find an approximation , which might result in original missing entries being replaced by new values (recommendations or ratings).

2. Do you just multiply? Yes. That it is the essence of calculation. For every user and product, multiplication gives you ratings or score. Sorting by score, and picking the index you get recommendation for each user. It also allows you to now store much smaller matrices than the original one.

3. Choice of factorization will be also dictated by application. If your application has only positive ratings, then it is better to use non-negative matrix factorization. You may start using matrix factorization methods without knowing the implementation to start with, as long as you are aware of the overall idea (and the pitfalls in using it).

Further Comments

It is a little perplexing that you start with a matrix with many missing entries (unseen items), approximate the matrix via factorization and expect to get non-missing entries (which help in doing predictions). If the task is to approximate original entries, then recommendations won't be good since you can have missing/zero entries in the approximation and still get lowest error on approximation task. The idea: regularization imposed in the algorithm (point 1 above) ensures that noise in the original data is filtered and only patterns are detected. But this idea has its fair share of critics. The introduction section of this paper by Stefen Rendle gives a readable introduction to what is happening in simple matrix factorization methods and what can go wrong. The paper also re-formulates the task with better optimization criteria. You can also read this post by Simon Funk which explains the mechanics in a readable language (and code), for matrix factorization applied to recommendations.