0
$\begingroup$

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function:

$$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1)$$

Questions:

  • what is the $c$ term? It is not explained in the documentation
  • What is the cost function minimized when LogisticRegression(multiclass=multinomial) is used instead?
$\endgroup$
1
$\begingroup$
  1. The c (small one) term is bias or intercept added to the model. This is similar to intercept we add in case of linear regression. The library allows you to set bias term to zero too.
  2. When set to multinomial model, the cost function will try to minimize cross-entropy loss. Hard for me to write down the equation here, but I always go back to this useful explanation of how cross entropy loss works and how it is minimized. In other words, a multinomial regression will work like a single layer neural network with logistic activation function.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.