# How would I be able to improve my CNN model (Keras)?

Recently I read a research paper on age detection using facial images. So right now because of that I was trying to see how far I could get by applying a CNN to a dataset of facial images (with their respective ages) in order to predict their ages which would be in bins (ex. 0-10, 11-20, 21-30...).

For training and testing

training.shape (50000, 28, 28)
testing.shape (2938, 28, 28)


I tried to keep the images small as they would be able to run faster as well as using grayscale. And for the actual layers themselves I tried to keep it simple, for now,

model = Sequential()

Layer (type)                 Output Shape              Param #
=================================================================
conv2d_1 (Conv2D)            (None, 26, 26, 64)        640
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 24, 24, 32)        18464
_________________________________________________________________
flatten_1 (Flatten)          (None, 18432)             0
_________________________________________________________________
dense_1 (Dense)              (None, 10)                184330
=================================================================
Total params: 203,434
Trainable params: 203,434
Non-trainable params: 0
_________________________________________________________________


Compiled with the following

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])


So far the best accuracy after running for 100 epochs has been 37.16. Which isn't great but recently I've gotten access to one of my schools gpu's so I wanted to fix anything that I'm doing wrong and improve my model. Is there anything you could recommend when it comes to improving the model, theres probably a lot this is more my first time trying to do this.

The first thing you should notice is that you've almost ruined your input signal. Take a look at a $$28\times28$$ image of a face? what can you see? is there any difference between a teenager and a middle-aged person? The point is that the network should be trained using data that does not have high Bayes error which means you as an expert can distinguish between inputs and label them correctly. Increase the size of your inputs. By doing so, if you use the current regime, you may have lots of trainable parameters between dense layers and convolutional layers. Consequently, try to employ more convolutional layers and some pooling layers among them. Also, try to add more dense layers with more neurons in each.

• Thank you thats actually great advice. Initially I was training on my laptop so I needed to decrease the size but now with gpu access I will do as you mentioned and reshape the images. But aside from that are there any specific reasons why a model say after 1 epoch would just drop in accuracy? Dec 13 '18 at 21:22
• You mean your accuracy was not what you expected after one epoch? If so, it may have different reasons. Your data may be difficult to be learned or your network is not large enough. There are more reasons. Dec 13 '18 at 21:24
• Yes, as I mentioned my best was 37ish but there would be some random times when I was training where by the second epoch my accuracy would be down to 17. Dec 13 '18 at 21:29
• You've not provided the training line of your code but I guess you've specified batch size. If you employ batch optimisation, your steps may not be exact moves toward downhill. Dec 13 '18 at 21:33
• That makes sense. I'll try that as well. Thank you again for you help @Media. Dec 13 '18 at 21:36

Two things come to my mind that you can try quickly.

1. Do NOT shrink your images, 28x28 is too small (it is fine for MNIST-like datasets, but not for faces). Also by making them grayscale you are missing a lot of valuable information, try to use color images.
2. Use a pretrained CNN, keras offers a number of them, I normally play quite a bit with VGG16 as it is a simple network to reuse. My recommendation is to freeze all the layers but the last one and see which performance you get (as a baseline). Then considering unfreezing other layers for increased performance.

Please do give it a shot to those options, as per the GPU, it is pretty much a MUST for CNNs, notice that kaggle.com is providing now jupyter notebooks with FREE GPU (it is on beta atm, but seems to do the job quite well).

• Thank you I'll also try this initially though I would like to see how far I could get my own CNN working but if it and it looks like it will comes down to using a pretrained model I'll do that. I didnt know about kaggles GPU enabled notebooks I'll check that out. Dec 13 '18 at 22:41
• My own experience is that it is not that easy to write your own CNN, it is often interesting to see other CNNs to get inspiration, the same way you do not implement your own hashtable, you do not write your own CNN (although I think is a very interesting exercise!) Dec 13 '18 at 23:19
• Do you have an reccomendations on how I would be able to reuse VGG16 in this instance? I have the images that i would like to train on as well as the target variables. The issue is if using VGG-16 im unsure as how to output results for my 10 labels as I expect the output to choose a age bin for a given image. Dec 14 '18 at 18:35