Logistic regression cost function is cross-entropy. It is defined as below:


This is a convex function. To reach the minimum, scikit-learn provides multiple types of solvers such as : ‘liblinear’ library, ‘newton-cg’, ‘sag’ and ‘lbfgs’.

Is it possible to analytically find the minimum? if yes, what can we say of the computing complexity?

  • 1
    $\begingroup$ Have you checked this post? It might be helpful. $\endgroup$
    – Shadi
    Nov 1, 2017 at 17:27

1 Answer 1


If you use just one neuron (linear->sigmoid), you can find the minimum with too much small error or maybe reach to the minimum using approaches like gradient descent or other optimization algorithms. The reason for finding the minimum is that cross entropy is a non-convex shape, but if you use sigmoid function as the activation function of logistic regression, the cross entropy cost function becomes convex and it is easy to find the only global minimum.

  • $\begingroup$ I am searching for an analytical solution expression with a closed form. Looking at Shadi answer, it seems that the solution can't be easily expressed for most of logistic regression problems (more than two input variables...). $\endgroup$
    – Theudbald
    Nov 26, 2017 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.