How do I interpret loss in a neural network?

Question

I am studying how to evaluate the performances of a convolutional neural network, and in particular I have seen that we have to look both at accuracy and loss. I don't understand why do we have to look also at the loss, and honestly I haven' t understand really clearly what the loss is. I have understood that it is something that we want to minimize and the lower the loss, the better the performances are.

But also I have seen that if the loss is to low we are overfitting, and also that there is a sort of relationship between loss and accuracy, but I have not clear what it really means.

To me this loss concept and how to interpret it both alone and with respect to the accuracy seems an abstract concept at the moment.

Can somebody help me clarify this concepts? Thanks in advance.

[EDIT] For example, I obtain this plot :

How do I interpret this?

From what I can see, it seems that the loss decreases both for train set and test set at the same rate, and we have that if this happens the model is good, otherwise we encouter overfitting or also underfitting. But I don't understand, why we have this? So why if the are close and decrease at same pace we are doing well, otherwise not?

Thanks.

Answer 1

Thinking in terms of optimisation, loss is convergence indicator

NN can be expressed as an optimization problem.

In the context of an optimization algorithm, the function used to evaluate a candidate solution (i.e. a set of weights) is referred to as the objective function. When we minimise this function we also refer to it as loss function

Cross-entropy and mean squared error are the two main types of loss functions to use when training neural networks.

Graph that you plotted indicates just convergance behavior on two different sets of our optimisation problem. If they diverge too much, we are overfitting. If they stop, we stopped learning.