5
$\begingroup$

I am studying how to evaluate the performances of a convolutional neural network, and in particular I have seen that we have to look both at accuracy and loss. I don't understand why do we have to look also at the loss, and honestly I haven' t understand really clearly what the loss is. I have understood that it is something that we want to minimize and the lower the loss, the better the performances are.

But also I have seen that if the loss is to low we are overfitting, and also that there is a sort of relationship between loss and accuracy, but I have not clear what it really means.

To me this loss concept and how to interpret it both alone and with respect to the accuracy seems an abstract concept at the moment.

Can somebody help me clarify this concepts? Thanks in advance.

[EDIT] For example, I obtain this plot :

enter image description here

How do I interpret this?

From what I can see, it seems that the loss decreases both for train set and test set at the same rate, and we have that if this happens the model is good, otherwise we encouter overfitting or also underfitting. But I don't understand, why we have this? So why if the are close and decrease at same pace we are doing well, otherwise not?

Thanks.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Loss function is simply a measure of error on each training example, while cost function is the average loss of the entire dataset. The goal is to minimize error to get a better fitting function. $\endgroup$ – serali Dec 6 '19 at 17:21
  • 1
    $\begingroup$ Reply on post,"datascience.stackexchange.com/questions/39825/…", bit helpful for you. $\endgroup$ – vipin bansal Mar 20 '20 at 7:46
  • 1
    $\begingroup$ This post and the link in it (and the links in that, and so on) may be of interest: stats.stackexchange.com/a/469059/247274. In short, accuracy has major flaws as a performance metric, as appealing as it first sounds. $\endgroup$ – Dave Aug 20 '20 at 2:18
0
$\begingroup$

Thinking in terms of optimisation, loss is convergence indicator

NN can be expressed as an optimization problem.

In the context of an optimization algorithm, the function used to evaluate a candidate solution (i.e. a set of weights) is referred to as the objective function. When we minimise this function we also refer to it as loss function

Cross-entropy and mean squared error are the two main types of loss functions to use when training neural networks.

Graph that you plotted indicates just convergance behavior on two different sets of our optimisation problem. If they diverge too much, we are overfitting. If they stop, we stopped learning.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ To add to this answer, the loss function essentially tells you how far the model's predictions are from the true values associated with the input. Here, as Noah as said in his answer, we use loss to optimise a neural network because we can back propagate and change the parameters of the model (weights and biases) with the respect the differences in the model's predictions and the true output values. $\endgroup$ – shepan6 Jun 20 '20 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.