Understanding Logistic Regression Cost function

Question

Linear Regression cost function:

$$J(\theta) = \frac{1}{2 m} \sum_{i=1}^m (h_{\theta}(x^{(i)}) - y^{(i)})^2$$

where:

$$h_{\theta}(x) = \theta_0 + \theta_1 x_1$$

Logistic Regression cost function

$$J(\theta) = \frac{1}{m} \sum_{i=1}^m - (y^{(i)} \times log(h_{\theta}(x{(i)})) + (1-y{(i)}) \times log(1 - h_{\theta}(x{(i)})))$$

where:

$$h_{\theta}(x) = g(\theta_0 + \theta_1 x_1 + \theta_2 x_2)$$

Intuitively linear regression is easy to understand as it optimizes the average squared distance between hypothesis and training data. But in case of logistic regression, I failed to understand the cost function.

What does the logistic regression cost function represent?

Answer 1

The way my intuition works for logistic regression is simple, say you are trying to classify if you have a picture with a dog on it, if that is the case, you output 1 (true), if not, you output 0 (false)

The cost function essentially represents the diff between what your model output (imagine, you have a dog in a picture, and your model outputs 0.75 instead of 1, your cost is 0.25).

Now, this is not a very mathematical way to put it, but at least it has helped me a lot to understand logistic regression.