2
$\begingroup$

I'm actually reading this tutorial about deepLearning and in particular about Logistic Regression.

I don't get why it first says to optimize logistic regression taking the max Probability and after using the Log loss function ?

Sorry you can explain me the point where him will use the Argmax and where Will use the Loss ?

you would not need only 1 of this 2 ?

$\endgroup$
2
$\begingroup$

The argmax is to get the class label prediction y_pred given a sample x.

# symbolic description of how to compute prediction as class whose # probability is maximal self.y_pred = T.argmax(self.p_y_given_x, axis=1)

The Loss function is used as optimization objective function to solve the coefficients W and b based on training data. On the tutorial, the loss function is the reverse the likelihood.

$\endgroup$
9
  • $\begingroup$ Worth reinforcing that the argmax function is not used to optimise anything - it is not really possible, because it is not differentiable. Instead it is being used to output a predictive value that we are interested in. There are ways to use it if for instance we want to meta-search parameters - then we might use accuracy on a cross-validation data set as a metric for the search. $\endgroup$ Jul 1 '16 at 6:58
  • $\begingroup$ @Xing Wang in the testing phase you don't get as output only 1 predicted value? why i have to do argmax ? $\endgroup$
    – T-student
    Jul 1 '16 at 9:30
  • $\begingroup$ @NeilSlater sorry meta serach you mean build a tree of possibile paramteres and get the best parameter ? and you want build this graph using the output of a cross-validation ? $\endgroup$
    – T-student
    Jul 1 '16 at 9:31
  • 1
    $\begingroup$ @Xing Wang i m not sure as ouput you get a vector of value and after you make the argmax ... in the testing phase i suppose to give a value and get as output 1 value... i dont get your point. After i learned all the W parameters after i should have as ouput a value. $\endgroup$
    – T-student
    Jul 1 '16 at 9:40
  • 1
    $\begingroup$ @T-student: Not usually a tree, but a vector of numeric options where you do not know how to set them to get best generalisation from your model. In logistic regression your search space for meta params is likely to be low dimensions, maybe just the single value for a regularisation param. So you would likely do a very simple search for the best accuracy (or F1 score, or logloss, or area under ROC, whichever metric you think best represents good solutions to your problem) using a cross-validation set, and just try a range of regularisation param values, training for each one in turn. $\endgroup$ Jul 1 '16 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.