Convolutional Neural Networks - When do I adjust parts of it, and how?

Question

I'm still training a Convolutional Neural Network model in Tensorflow to recognize age groups from facial images. I'm using the Adience 3D (frontalized with a 3D model - it might not be good data) dataset, with four folds for training, and one fold for testing.

I'm currently using three Conv-Conv-Pool layer sets, the first one's Conv layer using 32 filters with 5x5 size, the second and third using 64 filters with 3x3 size, and to top it all off, a Dense layer with flattened input, then a Dropout layer, dropping out the outputs of the dense layer by 20%, then on to the logits layer.

I normalized my cropped color images with the per-channel normalization - the (A - np.mean(A)) / np.std(A) method for each 64x64 array in the 64x64x3 input.

Currently I'm training it with Batch Size 64 and 100 steps so as to evaluate the weights every 100 steps, and while I don't see much improvement for now, I could see it increasing from steps 100 to 500 (0.27 to 0.35), before dropping on stage 600 then going up again up to step 800 (0.30 - 0.34).

Is this a sign I have to change my layer structure?

On that note, when should I change up my layer structure/hyperparameters like training steps, batch size, filter size, etc., and how?

Answer 1

If you wanna get the optimal parameters, you can try using grid search. A grid search or parameter sweep does an exhaustive searching through a specified subset (you have to define this subset) of the hyperparameter space of the learning algorithm which in your case is the Convolutional Neural Network. Although this may take a long time but this is one of the ways you'll know the good parameters for your model.

Try searching for optimization algorithms or parameter tuning techniques.