I need some help in the assessment of the training results of a convolutional neural network. Here is my setup:
- Architecture: InceptionV3
- Pre-trained InceptionV3 with weights from image net
- replaced last layer, retrained the second half of the network
- Classification: Softmax with cross entropy ploss
- Adam optimizer with keras default parameters
- Hyper parameters: Learning rate 0.0001, batch size 64
- Goal: Multi-label classification with exclusive labels
- Data: 17 classes ranging from 0-16 with 48.600 images in total
- Train, Test, Validation: 80%, 10%, 10%
- Training: 2 epochs training only last layer + 2 epochs training second half of network
Based on my validation set I created a confusion matrix with the following results.
Here are also some more per-class performance metrics:
As you can see, performance is not very good. The test-accuracy was at 62.2% so it was also not very good. Looking at the confusion matrix it is noticeable that the classes 0, 2, 3, 7, 10, 11, 13 and 14 are never predicted. These classes have also the smallest number of samples. I wonder why this happens? Those classes are more rare than the others but nver predicting them seems odd. What would you recommend going forward?