1
$\begingroup$

I'm trying to build a neural network for an age detection task. Here some details :

  1. Dataset: I am using the "facial age" Kaggle dataset and the "UTKFace" dataset for a total of about 35k images I've divided the total dataset in train, dev and test set (70%-15%-15%) and I've applied data augmentation on the train set
  2. Task: Age detection, 8 class classification problem
  3. Model: I've applied transfer learning on ResNet-50, keras implementation

Model:

enter image description here

Results:

enter image description here

enter image description here

On the test set the model return an accuracy of 85.5% but on real world images (google images or personal photos) the model perform a lot worse (about 45%). The images of the real world I've choosen are quite similar to those of the datasets, it's not possible for an human eye to distinguish them.

Where is the problem ? What can I do to fix it ?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ To me it's suspicious that the validation accuracy is significantly higher than the training one. I would suspect something messed up in the data. $\endgroup$
    – Erwan
    Nov 3, 2022 at 23:27
  • $\begingroup$ Agree with @Erwan. BTW I'd suggest manually running some test images through the model and inspect the results manually for any weird thing. $\endgroup$
    – lpounng
    Nov 4, 2022 at 4:17
  • $\begingroup$ @Erwan From what I know about this situation, it is because of the use of many Dropout layers. In the training phase a certain percentage of features are not considered, depending on the Dropout layer, instead in the testing phase all features are always used so the model is more robust and can lead to higher accuracy. I don't think it's a completely weird situation and I'm not sure this is the problem. discussion on val acc > train acc. $\endgroup$
    – Daniel_Fortesque
    Nov 9, 2022 at 14:41
  • $\begingroup$ @Erwan I've checked the data many times but I can't find any strange thing. Is it possible that the problem is only the difference between the datasets I used and the real data ? $\endgroup$
    – Daniel_Fortesque
    Nov 9, 2022 at 14:45
  • $\begingroup$ @lpounng What do you mean specifically? I've visualized some test and real images and they seems very similar. I've also runned some test images one at a time through the model and the results are consistent. $\endgroup$
    – Daniel_Fortesque
    Nov 9, 2022 at 14:53

1 Answer 1

Reset to default
0
$\begingroup$

  1. As per @Erwan, it is suspicious that the validation accuracy is significantly higher than the training one. Try K-fold validation and see if this is true for all folds; if they all show this kind of behavior, there is something seriously wrong with your data/code.

  2. Double-check if you have shuffle dataset before/while splitting into train/val/test. Chance is a lot of 'easy' images have fallen into validation/test set.

  3. The 'real world' images may have some properties/distributions which are different from training data, yet undetected by naked eyes. If you have the resources, try labeling a few batches of real world images, append to the training set, and see if it impacts (lowers) the accuracy during training.

P.S.: please, do not post code as images.

Improve this answer
$\endgroup$
3
  • $\begingroup$ 1. As I mentioned in the comments it's because of the Dropout layers, it's not a weird situation. discussion on val acc > train acc 2. I've shuffled the dataset before splitting it into train/val/test 3. I'll try your suggestion, but it's not easy to get a high number of face images from web. I have collected 300 images manually from web but I don't know if they can have an impact on a training set of ~25k images $\endgroup$
    – Daniel_Fortesque
    Nov 10, 2022 at 11:56
  • $\begingroup$ Forgive me, but if you keep insisting 'nothing's wrong', 'cannot do', there is no way we can help nor explain why your test accuracy is low. Be skeptical and assume we are wrong to start with - this is the heart of science. $\endgroup$
    – lpounng
    Nov 11, 2022 at 17:10
  • $\begingroup$ If the set of real-world data is too small, you may try oversampling or assigning a heavier weight to them. $\endgroup$
    – lpounng
    Nov 16, 2022 at 9:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.