0
$\begingroup$

I really do not understand what does this code do

M = sparse.coo_matrix(([1]*n, (Y, range(n))), shape=(k,n)).toarray()

Cost function The code is related to calculating the sparse function in this equation, but I am really confused and I do not know how it iterates through it and what is: 1- sparse.coo_matrix 2- (Y, range(n))) 3-shape=(k,n)).toarray() ??

Also, What exactly does this term means in the equation and how to interpret it into code:enter image description here

Thank you , and please forgive my poor English.

$\endgroup$
0
2
$\begingroup$

It's a loss function applied to a regression with l2 penalty on the parameters. The first square brackets can be interpreted in the following way:

  • $ - \frac{1}{n} $ has the minus because it wants to minimize.
  • $\sum_{i=1}^{n}$ means for each data point.
  • $\sum_{j=0}^{k-1} $ means for each class.
  • $y_i == j$ means that the fraction after this term is calculated only for true class. This check makes sense because y true is an OHE vector like [0, 0, 1], therefore you want to evaluate only the predicted probability associated with the true class. Take into account softmax function: if you increase the probability of a single output in output in the softmax you are implicitly reducing the probabilities of the other outputs.
  • the log division is the softmax function.
  • $ + \frac{\lambda}{2} \dots $ is a l2 regularization term on the model parameters.
$\endgroup$
3
  • 1
    $\begingroup$ Thank you very much Mikedev for your help, would you please explain the attached code elements too ? and what does OHE stand for? and does the result of y_i == j is multiplied by the log function next to it ? thanks again I really appreciate your help $\endgroup$ Oct 19 '20 at 19:33
  • $\begingroup$ OHE = one hot encoding. $y_i$ is the label of ith row of the dataset like 3, 4 etc. (a discrete number). Therefore y_i == j means that you evaluate in the loss function only the jth output of the model that corresponds the probability of class j $\endgroup$
    – Mikedev
    Oct 19 '20 at 21:43
  • $\begingroup$ Thank you very much Mikedev, would you please explain the attached code as well ? $\endgroup$ Oct 19 '20 at 23:41

Not the answer you're looking for? Browse other questions tagged or ask your own question.