Pytorch: Starting with a high loss value, but the loss converged at the end. I dont know if the model could start with a loss > 100. Help!

Question

I have been trying to attempt plant disease detection using transfer learning methods. I chose ResNet50 first. I also performed a baseline model which is a CNN model. In resnet50, I used cross entropy loss and trained the model for 30 epochs. I did batch normalization too. Initially epoch 1 loss was 112.5250 and the training loss was 87.512. But, for the last epoch it was 2.1660 and Validation loss was 1.8905 with Validation accuracy as 0.995. The overall accuracy of the model was 98.8% and the model doesn't seem to overfit too.

Before training the model, I performed hyper parameter optimization where I optimized Learning rate and momentum using Bayesian optimization. I optimized weight decay using Cross validation. I performed batch normalization. Before that, I trained the model without any optimization. I just assumed the learning rate to be 0.001, same loss function and trained for 30 epochs. In this case, the model started with a loss value of 114.2 and validation loss was 130.46. It converged to 9.9038 with accuracy of 97%.

So, my question is, it is possible for the model to start with such a high loss value despite giving a good accuracy at the end? Does the magnitude of the loss have nothing to do with the correctness of the model nor the accuracy? If the model started with a huge loss, but gave a good accuracy, is my model bad?

First 5 epochs

EPOCH: 1 

Epoch time taken: 28.61398434638977 seconds. Epoch 1 loss: 118.5250 Validation loss: 87.5214 Validation accuracy: 0.753

EPOCH: 2 

Epoch time taken: 28.737553358078003 seconds. Epoch 2 loss: 68.8141 Validation loss: 50.3452 Validation accuracy: 0.858

EPOCH: 3 

Epoch time taken: 28.45273518562317 seconds. Epoch 3 loss: 41.5181 Validation loss: 31.1520 Validation accuracy: 0.928

EPOCH: 4 

Epoch time taken: 28.434630155563354 seconds. Epoch 4 loss: 26.8844 Validation loss: 21.1203 Validation accuracy: 0.948

EPOCH: 5 

Epoch time taken: 28.762181282043457 seconds. Epoch 5 loss: 19.3646 Validation loss: 15.4843 Validation accuracy: 0.955

Last 5 epochs

EPOCH: 26 

Epoch time taken: 28.35645818710327 seconds. Epoch 26 loss: 2.8168 Validation loss: 2.2771 Validation accuracy: 0.990

EPOCH: 27 

Epoch time taken: 28.51978588104248 seconds. Epoch 27 loss: 3.3986 Validation loss: 2.2274 Validation accuracy: 0.992

EPOCH: 28 

Epoch time taken: 28.51390767097473 seconds. Epoch 28 loss: 2.2707 Validation loss: 1.9976 Validation accuracy: 0.992

EPOCH: 29 

Epoch time taken: 28.456573247909546 seconds. Epoch 29 loss: 2.5344 Validation loss: 1.9272 Validation accuracy: 0.990

EPOCH: 30 

Epoch time taken: 28.486974239349365 seconds. Epoch 30 loss: 2.1660 Validation loss: 1.8905 Validation accuracy: 0.995
```

Answer 1

The value of the loss in the first epochs is irrelevant. The network weights are initialized at random, so it is perfectly expected that the behavior of the network differs from the desired one.

However, this does not mean that you should only look at the last value of the training and validation loss. Instead, you should plot the whole curves, to understand the training dynamics. From the curves, you can see whether your network overfitted or underfitted, if there's still room for improvement if you train the model for longer, etc.

There are many questions in this site asking about how to interpret training/validation curves (e.g. this and this). You may take a look at them to understand how to interpret yours. From the values you posted, it seems there's still room for improvement if you train for more epochs.