I participate in the development of a matrix factorization algorithm and I have some convergence issues. Here is the kind of minimization problem I am facing :

$L(D,A) = ||X-DA||^2 + \sum_{(i,j) \in E} (A_{i} - A_{j})^2$

with the following constraints $\forall i \in (1,..,n), X_i \geq 0$ and $\sum_{i} A_{i} = 1$

$X \in R^{n \times m} $ is the dataset (a flattened version of a 3D dataset). $D \in R^{n \times k}$, $A \in R^{k \times m}$ and $E$ is the set of neighboring elements in the original 3D version of X.

To solve this biconvex optimization problem, an alterneted gradient descent projected on the constraints was chosen. The projection on the constraints is done using the proximal method (Proximal_gradient_method). If i understood correctly, this algorithm should theoretically converge towards a second-order stationary point (Li, Q. et al. (2019, May). Internat. Conf. Machine Learning (pp. 3935-3943).).

In practice, I observe that the residuals do not only contain noise as some features of X were not fitted by the algorithm. I suspect that there is some computing issues. However, I am quite new to the field and I am searching for advices on how to debug such algorithm and make sure that it converged. I could not find practical advices in the literature to solve these issues. Is there any book, scientific article, etc... on how to check the convergence of my algorithm ?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.