# What is the efficiency difference between different cost functions in case of neural networks?

I'm studying the theory behind neural nets and I wonder if there is any actual difference between using different lost/cost functions?

Let's say I could use either MAE or MSE for backpropagating loss; does this decision have any actual effect on the model efficiency?

At the end of the day both functions just calculate the difference between $y$ and $\hat y$ (albeit on a different scale). But for the optimizer, only the error tendency is what really matters, not the absolute difference.

Of course, this question is relevant for any other model evaluation.

• This has nothing to do with neural networks per se. Try it and see what difference it makes! Note that you are implicitly assuming a regression task. And how are you defining "efficiency"? – Emre Aug 27 '17 at 18:05