2
$\begingroup$

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$

The other one is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\Big[y^{i}\log(a^{i})+(1-y^{i})\log(1-a^{i})\Big]$$

As I can see those two equations are not equal. How both can be used to calculate cost function?

Also, one of them using $h$ function which is $a$ of output layer, whereas others are using $a$ ($a$ is $f(w*x)$ where $f$ is activation function). When I looked from the book "Pattern Recognition and Machine Learning" from Bishop, he used $a$ for both of the equations. But from another source which I took equations from $h$ is used. But using different $a$ values and using just one of them (namely $h$ which is $a$ of output) are totally different things.

Both sources are reliable, what am I missing?

Improve this question
$\endgroup$
4
  • $\begingroup$ You aren't missing anything imho, that's just different way to write eqns. (I believe one is vectorized and other one is having double sigma(looping over individual items)) will get back to this. $\endgroup$
    – Aditya
    Jul 15, 2019 at 2:53
  • $\begingroup$ I didn't give much attention because I usually use vectorized version but what is $$h(x^{i})$$ if $$h(x)=(theta*a^{l-1})$$ where l is total number of layers? what it becomes equal to? $\endgroup$
    – J.Smith
    Jul 15, 2019 at 3:20
  • $\begingroup$ Are you sure that the second sum in the first formula is supposed to be $\sum_{k=1}^k$? $\endgroup$
    – Elias Strehle
    Jul 15, 2019 at 13:40
  • $\begingroup$ @EliasStrehle Capital k, not lowercase. Sorry. $\endgroup$
    – J.Smith
    Jul 15, 2019 at 15:59

2 Answers 2

Reset to default
0
$\begingroup$

I don't remember exactly what the book has mentioned, but I guess the difference between the two is due to having one or multiple features. I guess it is already mentioned in the book. They are the same. One is for multi-dimensional input, and the other is for one-dimensional. One sigma is iterating over the features and the other is iterating over the examples. You can put $k$ to one to achieve the simpler formula.

Improve this answer
$\endgroup$
4
  • $\begingroup$ You changed your handler! (Comment will be deleted) $\endgroup$
    – Aditya
    Jul 15, 2019 at 5:27
  • $\begingroup$ @Aditya opppsss :D $\endgroup$
    – Green Falcon
    Jul 15, 2019 at 5:45
  • $\begingroup$ @Vaalizaadeh Yes, that is for two sums. But how about the uses of 'h' and 'a'? If you know the answer can you add it to yours? $\endgroup$
    – J.Smith
    Jul 15, 2019 at 15:58
  • $\begingroup$ @J.Smith It is just a naming convention, $h_{\theta}(x)$ = $a$. $a$ is considered as the activation of last layer. $\endgroup$
    – Green Falcon
    Jul 16, 2019 at 5:14
0
$\begingroup$

ideally cost functions are used to identify the global minima(where the error value is least) for all the input values. what so ever the cost function it will only for check for least value, using different methods but value remain same for all cost function.

cost function graph

in the above image the value which is about to touch the x axis that is the best cost values and that remains same in all cost functions.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.