Recently I've been working on a mini side project in detecting age off of facial images. Aside from mistakes, I have made decent progress in creating my model.
(50000, 120, 120) - training
(2000, 120, 120) - testing
My current updated model is the following
model = Sequential()
model.add(Conv2D(64, kernel_size=3, use_bias=False,
input_shape=(size, size, 1)))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Conv2D(32, kernel_size=3, use_bias=False))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2, 2)))
#model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, use_bias=False, kernel_regularizer=regularizers.l2(
0.01)))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Dense(64, use_bias=False, kernel_regularizer=regularizers.l2(
0.01)))
model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
#keras.utils.plot_model(model, to_file='test_keras_plot_model.png',show_shapes=True)
adam = optimizers.adam(lr=0.001)
model.compile(optimizer=adam, loss='categorical_crossentropy',
metrics=['accuracy'])
and when training on 50 epochs my results are as shown
Train on 50000 samples, validate on 2939 samples
Epoch 1/50
50000/50000 [==============================] - 152s 3ms/step - loss: 2.8988 - acc: 0.3521 - val_loss: 2.5428 - val_acc: 0.3855
Epoch 2/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.3828 - acc: 0.3830 - val_loss: 2.3649 - val_acc: 0.3828
Epoch 3/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.3165 - acc: 0.3854 - val_loss: 2.3128 - val_acc: 0.3831
Epoch 4/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.2195 - acc: 0.3878 - val_loss: 2.1559 - val_acc: 0.3930
Epoch 5/50
50000/50000 [==============================] - 148s 3ms/step - loss: 2.1517 - acc: 0.3922 - val_loss: 2.2298 - val_acc: 0.3831
Epoch 6/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.1291 - acc: 0.3984 - val_loss: 2.2902 - val_acc: 0.3882
Epoch 7/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.1371 - acc: 0.3993 - val_loss: 2.2068 - val_acc: 0.3627
Epoch 8/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.0940 - acc: 0.4025 - val_loss: 2.2656 - val_acc: 0.3838
Epoch 9/50
50000/50000 [==============================] - 148s 3ms/step - loss: 2.0475 - acc: 0.4062 - val_loss: 2.2142 - val_acc: 0.3848
Epoch 10/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.0358 - acc: 0.4081 - val_loss: 2.0059 - val_acc: 0.4052
Epoch 11/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.9894 - acc: 0.4131 - val_loss: 2.0070 - val_acc: 0.4117
Epoch 12/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9701 - acc: 0.4122 - val_loss: 2.1154 - val_acc: 0.3995
Epoch 13/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9273 - acc: 0.4146 - val_loss: 1.9478 - val_acc: 0.4151
Epoch 14/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9040 - acc: 0.4170 - val_loss: 1.8918 - val_acc: 0.4226
Epoch 15/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.8449 - acc: 0.4171 - val_loss: 1.9196 - val_acc: 0.4124
Epoch 16/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.8331 - acc: 0.4172 - val_loss: 1.8900 - val_acc: 0.4114
Epoch 17/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.8088 - acc: 0.4179 - val_loss: 1.7958 - val_acc: 0.4195
Epoch 18/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7912 - acc: 0.4194 - val_loss: 1.7635 - val_acc: 0.4246
Epoch 19/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7748 - acc: 0.4211 - val_loss: 1.7244 - val_acc: 0.4274
Epoch 20/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7486 - acc: 0.4217 - val_loss: 1.7267 - val_acc: 0.4311
Epoch 21/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7321 - acc: 0.4233 - val_loss: 1.7271 - val_acc: 0.4226
Epoch 22/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7187 - acc: 0.4242 - val_loss: 1.7352 - val_acc: 0.4212
Epoch 23/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7151 - acc: 0.4232 - val_loss: 1.7118 - val_acc: 0.4195
Epoch 24/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7045 - acc: 0.4241 - val_loss: 1.6968 - val_acc: 0.4233
Epoch 25/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6970 - acc: 0.4251 - val_loss: 1.6989 - val_acc: 0.4182
Epoch 26/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6908 - acc: 0.4265 - val_loss: 1.6868 - val_acc: 0.4233
Epoch 27/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6904 - acc: 0.4258 - val_loss: 1.7385 - val_acc: 0.4083
Epoch 28/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6885 - acc: 0.4249 - val_loss: 1.7458 - val_acc: 0.3974
Epoch 29/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6911 - acc: 0.4265 - val_loss: 1.7251 - val_acc: 0.4093
Epoch 30/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6895 - acc: 0.4280 - val_loss: 1.8342 - val_acc: 0.4008
Epoch 31/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6877 - acc: 0.4288 - val_loss: 1.6965 - val_acc: 0.4161
Epoch 32/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6799 - acc: 0.4275 - val_loss: 1.7304 - val_acc: 0.4110
Epoch 33/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6869 - acc: 0.4294 - val_loss: 1.6955 - val_acc: 0.4250
Epoch 34/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6827 - acc: 0.4286 - val_loss: 1.7588 - val_acc: 0.4158
Epoch 35/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6863 - acc: 0.4286 - val_loss: 1.7112 - val_acc: 0.4188
Epoch 36/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6856 - acc: 0.4304 - val_loss: 1.6767 - val_acc: 0.4240
Epoch 37/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6878 - acc: 0.4293 - val_loss: 1.7029 - val_acc: 0.4246
Epoch 38/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6835 - acc: 0.4325 - val_loss: 1.6990 - val_acc: 0.4110
Epoch 39/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6821 - acc: 0.4307 - val_loss: 1.7143 - val_acc: 0.4274
Epoch 40/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6866 - acc: 0.4300 - val_loss: 1.7052 - val_acc: 0.4226
Epoch 41/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6880 - acc: 0.4322 - val_loss: 1.7299 - val_acc: 0.4137
Epoch 42/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6847 - acc: 0.4322 - val_loss: 1.8020 - val_acc: 0.4008
Epoch 43/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6805 - acc: 0.4301 - val_loss: 1.7149 - val_acc: 0.4175
Epoch 44/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6840 - acc: 0.4325 - val_loss: 1.7346 - val_acc: 0.4158
Epoch 45/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6876 - acc: 0.4340 - val_loss: 1.8701 - val_acc: 0.3909
Epoch 46/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6890 - acc: 0.4315 - val_loss: 1.6840 - val_acc: 0.4277
Epoch 47/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6870 - acc: 0.4334 - val_loss: 1.7338 - val_acc: 0.4219
Epoch 48/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6895 - acc: 0.4318 - val_loss: 1.7242 - val_acc: 0.4246
Epoch 49/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6877 - acc: 0.4366 - val_loss: 1.8392 - val_acc: 0.4001
Epoch 50/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6844 - acc: 0.4347 - val_loss: 1.7266 - val_acc: 0.4192
Model Saved
There are still ways that I can obviously improve my model and currently I am working on data augmentation to improve my accuracy while also dealing with some over-fitting issues.
But I just wanted to know in general what are some aspects of creating a cnn model should one keep in mind? As well as tips/tricks for creating improvement in results? Also if you have any advice or critiques on my current model any input would be appreciated (i'm still working on improving it) -- thank you