# Understanding Logistic Regression Cost function

Linear Regression cost function:

$$J(\theta) = \frac{1}{2 m} \sum_{i=1}^m (h_{\theta}(x^{(i)}) - y^{(i)})^2$$

where:

$$h_{\theta}(x) = \theta_0 + \theta_1 x_1$$

Logistic Regression cost function

$$J(\theta) = \frac{1}{m} \sum_{i=1}^m - (y^{(i)} \times log(h_{\theta}(x{(i)})) + (1-y{(i)}) \times log(1 - h_{\theta}(x{(i)})))$$

where:

$$h_{\theta}(x) = g(\theta_0 + \theta_1 x_1 + \theta_2 x_2)$$

Intuitively linear regression is easy to understand as it optimizes the average squared distance between hypothesis and training data. But in case of logistic regression, I failed to understand the cost function.

What does the logistic regression cost function represent?

• It's negative log likelihood, so if you minimize this function, you consequently maximize likelihood of the data. They derive it for the case with 2 classes here. – nikkou Aug 26 '17 at 7:26
• Likelihood function in this case is $\prod h_\theta (x^{(i)})^{y^{(i)}} (1 - h_\theta (x^{(i)})^{(1-y^{(i)})}$. Negative log likelihood is equal to your cost function (except for the multiplier). – nikkou Aug 26 '17 at 7:33
• Atinesh: In this formulation $y^{(i)} \in \{0,1\}$, so either way one of the terms gets eliminated. It's a succint way of expression of defining this piecewise function. When you take the logarithm the products separate into sums. – Emre Aug 26 '17 at 8:16
• I acknowledge it is not that compliant to post a link as an answer, but for the case in point the following video by Andrew Ng which can really help original poster youtube.com/watch?v=HIQlmHxI6-0 – tagoma Aug 27 '17 at 7:45