I just finished training two models, while the one is pretrained and the other trained from scratch and created two diagrams afterward with their data, but as I am very new to machine learning, I don't get what they state.

Why is the training accuracy so low? Did I use to less data? I had about 7200 pictures for training and 800 for validation!

enter image description here

What does it mean, that the validation accuracy of the pretrained algorith is so much higher as the other one? Does it mean the pretrained is two times better then the one trained from scratch?

enter image description here

  • 2
    $\begingroup$ I have so many questions after reading your question. Nothing seems to make sense. May we have a little more information about your problem ? (is it classification, how many classes ?). Are the curves on your first graph mixed up ? (having a validation accuracy better than the training accuracy is giga weird). What model are you using ? what optimizer are you using ? Is it Keras or Pytorch ? $\endgroup$
    – Ubikuity
    Jun 8 '21 at 17:30
  • $\begingroup$ Sorry, i don't know much about CNNs and machine learning. It is binary classification that should detect placeholder images in a bunch of product pictures. I think the data is just very bad $\endgroup$
    – lxg95
    Jun 8 '21 at 17:35
  • $\begingroup$ Do let me know if you are satisfied with the answer? If not I will try my best possible way to edit it. Please consider accepting the answer if it answers your question. $\endgroup$ Jun 14 '21 at 14:26

Why is the training accuracy so low?

This is because your model is underfit. Few of the reasons for this could be,

  1. you might be using small learning rate.
  2. your model architecture is simple (small) and not big enough to recognize patterns from the data. Try increasing layers.
  3. try removing regularization if any.

As per the best of my knowledge and assumptions, I think following could be some of the reasons for validation accuracy to be higher than training accuracy. You might consider investigating in these areas.

  1. The dataset domain might not be consistent? This means that there might be different types of images present in you dataset. For some images (or a type of images) the model is able to learn correctly (as a result ~ 50% accuracy on train set). And for the rest of the images the model is getting confused i.e. it is difficult for the model to recognize these other 50% images. And there might be a possibility that the particular type of images that are easier for model to recognize are present in the validation set. You can ensure that the domain of train and validation sets is same.

  2. The dataset might not be properly split? This means the domain might be consistent but the dataset has imbalanced classes? There might be a possibility that the train set might contain some classes having more instances (majority classes) and some classes having very less instances (minority classes). Generally, model gets a hard time recognizing these minority classes, hence less train accuracy. And perhaps the validation set is containing only majority classes, which are very easy for the model to recognize.

What does it mean, that the validation accuracy of the pretrained algorith is so much higher as the other one? Does it mean the pretrained is two times better then the one trained from scratch?

Yes, it means given unseen 800 images to both of the models, the pretrained model predictions are two times better then the one trained from scratch.

Edited as per the suggestion from Nikos M.

  • $\begingroup$ you should provide an explanation why validation accuracy is high(er than training accuracy) in the first point. Underfitting as your answer suggests, in principle should have both of them low. Not only training accuracy $\endgroup$
    – Nikos M.
    Jun 8 '21 at 17:55
  • $\begingroup$ Thank you for pointing that out, edited my answer. $\endgroup$ Jun 8 '21 at 19:28
  • $\begingroup$ Hmm, I dont think these reasons are sufficient, but the OP will judge that $\endgroup$
    – Nikos M.
    Jun 8 '21 at 19:29
  • 1
    $\begingroup$ Yes might be a case. But even if the model is pretrained, you are fine-tuning it anyways. So training on the garbage data should have affected the validation accuracy, which is not happenning. Lastly, if you are satisfied with my answer you can go ahead and accept it. Or do let me know, I will try my best to edit my answer. $\endgroup$ Jun 9 '21 at 14:09
  • 1
    $\begingroup$ It helped me a lot in understanding my model, sry that it took so long until i accepted it. Thank you very much :) $\endgroup$
    – lxg95
    Jun 15 '21 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.