Math of Logistic regression cost function

Question

In the current scikit-learn documentation for binary Logistic regression there is the minimization of the following cost function:

$$\min_{w, c} \frac{1}{2}w^T w + C \sum_{i=1}^n \log(\exp(- y_i (X_i^T w + c)) + 1)$$

Questions:

• what is the $c$ term? It is not explained in the documentation
• What is the cost function minimized when LogisticRegression(multiclass=multinomial) is used instead?

Answer 1

1. The c (small one) term is bias or intercept added to the model. This is similar to intercept we add in case of linear regression. The library allows you to set bias term to zero too.
2. When set to multinomial model, the cost function will try to minimize cross-entropy loss. Hard for me to write down the equation here, but I always go back to this useful explanation of how cross entropy loss works and how it is minimized. In other words, a multinomial regression will work like a single layer neural network with logistic activation function.