I’ve playing with the MovieLens ratings dataset under Spark’s ALS and a manual implementation of ALS and comparing results with the same hyperparameters. I’d like to know this exactly in order to make some experiments that could be reproducible. I’ve seen several papers describing the exact loss formula for ALS in different ways. These seem to be quite common: $$1)\: min \sum (r_{ui} - \mu – b_u – b_i – p_u^Tq_i)^2 + \lambda (\lVert \mathbf{p_u} \rVert^2 + (\lVert \mathbf{q_i} \rVert^2 + b_u^2 + b_i^2)$$ (The ratings are centered by subtracting an overall bias $\mu$, and for each user, a bias $b_u$ which is a part of the model, and part of the regularization, and a similar bias for each movie $b_i$)

$$2)\: min \sum (r_{uj} – u_i^Tm_j)^2 + \lambda (n_{u_{i}} \lVert \mathbf{ u_i } \rVert^2 + n_{m_{j}} \lVert \mathbf{m_j} \rVert^2)$$

(The penalization for each user-feature and movie-feature vector are multiplied by the number of ratings from each user/movie, and the ratings might be centered beforehand, but the biases are not model parameters).

And I’ve also seen other variations where the l2-norms of the vectors are not squared.

I’ve playing with different variations of these formulas and comparing the results on the same data with Spark’s implementation and my manual implementation, and I cannot find any variation which would make the same hyperparameters in both implementations (the number of latent factors and the value of lambda) give similar results with the same data (e.g. under some formulas, with the same number of latent factors and using the same value for lambda that in Spark gives good cross-validated results, in my implementation results in either overfitting badly or making all the predictions close to zero, but a totally different value of lambda gives comparable results).

What formula does Spark use? Also, how are the initial matrices initialized? I’ve seen some suggestions saying to let the first column of the movie-factor matrix be the average rating for the movie and the rest random-normal numbers, while others suggest random uniform numbers with quite big signs. How is the first movie-factor matrix initialized?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.