I am working on collaborative filtering using matrix factorization in MATLAB. I am using Gradient Descent for parameter learning. The cost function to optimize is :

$ J = {\left\| I \odot (R - U V') \right\|}_{F}^{2} + \lambda_1 {\left\| U \right\|}_{F}^{2} + \lambda_2 {\left\| V \right\|}_{F}^{2} $

I have the following equations for updates :

$ \nabla J_{u(i)} = -\sum_j I_{ij} (r_{ij} - u_i v_j') v_j $

$ \nabla J_{v(i)} = -\sum_j I_{ij} (r_{ij} - u_i v_j') u_i $

The dimensions of the matrices are :

$I -> n * n$

$R -> n*n$

$U -> n*k$

$V -> n*k$

I am not able to come up with vectorized form of the gradient, so I have to loop over i's and j's explicitly, which is slowing down the code a lot.

  • 1
    $\begingroup$ 1) This kind of looks like one of Andrew NG's homework questions... is it? 2) What is the F subscript? $\endgroup$
    – AN6U5
    Jul 24 '16 at 15:50
  • $\begingroup$ Also, 3) Is the "out of the page arrow" meant to be a vector dot product? 4) Why are you taking the lengths of vectors before squaring them? Its the same thing as just squaring them and will slow your code down. $\endgroup$
    – AN6U5
    Jul 24 '16 at 15:57
  • $\begingroup$ How did you derive the gradients in the first place? Derive them in matrix form and voila. $\endgroup$
    – Jim
    Sep 22 '16 at 19:54

Is your question how to vectorize the equations for updating? You have a vector of i and j, and for each update you are summing over j. Calculate the vectors of each of the components and then sum the vector over j.

Providing actual code might help your question. As is, it seems vague.

  • $\begingroup$ Yeah, I am trying to vectorize the update equations for ui and vj. As you pointed out correctly, for each i I am summing over all j, which is very costly. I want to vectorize ui. $\endgroup$ Jun 24 '16 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.