# Two different cost functions for neural networks, how they can give the same result?

One is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}\Big[y_{k}^{i}\log\big((h_{\theta}(x^{i}))_k\big)+(1-y_{k}^{i})\log\big(1-(h_{\theta}(x^{i}))_k\big)\Big]$$

The other one is: $$J=-\frac{1}{m}\sum_{i=1}^{m}\Big[y^{i}\log(a^{i})+(1-y^{i})\log(1-a^{i})\Big]$$

As I can see those two equations are not equal. How both can be used to calculate cost function?

Also, one of them using $$h$$ function which is $$a$$ of output layer, whereas others are using $$a$$ ($$a$$ is $$f(w*x)$$ where $$f$$ is activation function). When I looked from the book "Pattern Recognition and Machine Learning" from Bishop, he used $$a$$ for both of the equations. But from another source which I took equations from $$h$$ is used. But using different $$a$$ values and using just one of them (namely $$h$$ which is $$a$$ of output) are totally different things.

Both sources are reliable, what am I missing?

• You aren't missing anything imho, that's just different way to write eqns. (I believe one is vectorized and other one is having double sigma(looping over individual items)) will get back to this. – Aditya Jul 15 '19 at 2:53
• I didn't give much attention because I usually use vectorized version but what is $$h(x^{i})$$ if $$h(x)=(theta*a^{l-1})$$ where l is total number of layers? what it becomes equal to? – J.Smith Jul 15 '19 at 3:20
• Are you sure that the second sum in the first formula is supposed to be $\sum_{k=1}^k$? – Elias Strehle Jul 15 '19 at 13:40
• @EliasStrehle Capital k, not lowercase. Sorry. – J.Smith Jul 15 '19 at 15:59

I don't remember exactly what the book has mentioned, but I guess the difference between the two is due to having one or multiple features. I guess it is already mentioned in the book. They are the same. One is for multi-dimensional input, and the other is for one-dimensional. One sigma is iterating over the features and the other is iterating over the examples. You can put $$k$$ to one to achieve the simpler formula.
• @J.Smith It is just a naming convention, $h_{\theta}(x)$ = $a$. $a$ is considered as the activation of last layer. – Media Jul 16 '19 at 5:14