0
$\begingroup$

Recently I've been working on a mini side project in detecting age off of facial images. Aside from mistakes, I have made decent progress in creating my model. 

(50000, 120, 120) - training
(2000, 120, 120) - testing

My current updated model is the following

    model = Sequential()

    model.add(Conv2D(64, kernel_size=3, use_bias=False,
                     input_shape=(size, size, 1)))
    model.add(BatchNormalization())
    model.add(Activation("relu"))

    model.add(Conv2D(32, kernel_size=3, use_bias=False))
    model.add(BatchNormalization())
    model.add(Activation("relu"))

    model.add(MaxPooling2D(pool_size=(2, 2)))
    #model.add(Dropout(0.25))
    model.add(Flatten())

    model.add(Dense(128, use_bias=False, kernel_regularizer=regularizers.l2(
        0.01)))
    model.add(BatchNormalization())
    model.add(Activation("relu"))

    model.add(Dense(64, use_bias=False, kernel_regularizer=regularizers.l2(
        0.01)))
    model.add(BatchNormalization())
    model.add(Activation("relu"))

    model.add(Dropout(0.5))
    model.add(Dense(10, activation='softmax'))

    #keras.utils.plot_model(model, to_file='test_keras_plot_model.png',show_shapes=True)

    adam = optimizers.adam(lr=0.001)
    model.compile(optimizer=adam, loss='categorical_crossentropy',
                  metrics=['accuracy'])

and when training on 50 epochs my results are as shown

Train on 50000 samples, validate on 2939 samples
Epoch 1/50
50000/50000 [==============================] - 152s 3ms/step - loss: 2.8988 - acc: 0.3521 - val_loss: 2.5428 - val_acc: 0.3855
Epoch 2/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.3828 - acc: 0.3830 - val_loss: 2.3649 - val_acc: 0.3828
Epoch 3/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.3165 - acc: 0.3854 - val_loss: 2.3128 - val_acc: 0.3831
Epoch 4/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.2195 - acc: 0.3878 - val_loss: 2.1559 - val_acc: 0.3930
Epoch 5/50
50000/50000 [==============================] - 148s 3ms/step - loss: 2.1517 - acc: 0.3922 - val_loss: 2.2298 - val_acc: 0.3831
Epoch 6/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.1291 - acc: 0.3984 - val_loss: 2.2902 - val_acc: 0.3882
Epoch 7/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.1371 - acc: 0.3993 - val_loss: 2.2068 - val_acc: 0.3627
Epoch 8/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.0940 - acc: 0.4025 - val_loss: 2.2656 - val_acc: 0.3838
Epoch 9/50
50000/50000 [==============================] - 148s 3ms/step - loss: 2.0475 - acc: 0.4062 - val_loss: 2.2142 - val_acc: 0.3848
Epoch 10/50
50000/50000 [==============================] - 149s 3ms/step - loss: 2.0358 - acc: 0.4081 - val_loss: 2.0059 - val_acc: 0.4052
Epoch 11/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.9894 - acc: 0.4131 - val_loss: 2.0070 - val_acc: 0.4117
Epoch 12/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9701 - acc: 0.4122 - val_loss: 2.1154 - val_acc: 0.3995
Epoch 13/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9273 - acc: 0.4146 - val_loss: 1.9478 - val_acc: 0.4151
Epoch 14/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.9040 - acc: 0.4170 - val_loss: 1.8918 - val_acc: 0.4226
Epoch 15/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.8449 - acc: 0.4171 - val_loss: 1.9196 - val_acc: 0.4124
Epoch 16/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.8331 - acc: 0.4172 - val_loss: 1.8900 - val_acc: 0.4114
Epoch 17/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.8088 - acc: 0.4179 - val_loss: 1.7958 - val_acc: 0.4195
Epoch 18/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7912 - acc: 0.4194 - val_loss: 1.7635 - val_acc: 0.4246
Epoch 19/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7748 - acc: 0.4211 - val_loss: 1.7244 - val_acc: 0.4274
Epoch 20/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7486 - acc: 0.4217 - val_loss: 1.7267 - val_acc: 0.4311
Epoch 21/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7321 - acc: 0.4233 - val_loss: 1.7271 - val_acc: 0.4226
Epoch 22/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7187 - acc: 0.4242 - val_loss: 1.7352 - val_acc: 0.4212
Epoch 23/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.7151 - acc: 0.4232 - val_loss: 1.7118 - val_acc: 0.4195
Epoch 24/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.7045 - acc: 0.4241 - val_loss: 1.6968 - val_acc: 0.4233
Epoch 25/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6970 - acc: 0.4251 - val_loss: 1.6989 - val_acc: 0.4182
Epoch 26/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6908 - acc: 0.4265 - val_loss: 1.6868 - val_acc: 0.4233
Epoch 27/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6904 - acc: 0.4258 - val_loss: 1.7385 - val_acc: 0.4083
Epoch 28/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6885 - acc: 0.4249 - val_loss: 1.7458 - val_acc: 0.3974
Epoch 29/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6911 - acc: 0.4265 - val_loss: 1.7251 - val_acc: 0.4093
Epoch 30/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6895 - acc: 0.4280 - val_loss: 1.8342 - val_acc: 0.4008
Epoch 31/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6877 - acc: 0.4288 - val_loss: 1.6965 - val_acc: 0.4161
Epoch 32/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6799 - acc: 0.4275 - val_loss: 1.7304 - val_acc: 0.4110
Epoch 33/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6869 - acc: 0.4294 - val_loss: 1.6955 - val_acc: 0.4250
Epoch 34/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6827 - acc: 0.4286 - val_loss: 1.7588 - val_acc: 0.4158
Epoch 35/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6863 - acc: 0.4286 - val_loss: 1.7112 - val_acc: 0.4188
Epoch 36/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6856 - acc: 0.4304 - val_loss: 1.6767 - val_acc: 0.4240
Epoch 37/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6878 - acc: 0.4293 - val_loss: 1.7029 - val_acc: 0.4246
Epoch 38/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6835 - acc: 0.4325 - val_loss: 1.6990 - val_acc: 0.4110
Epoch 39/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6821 - acc: 0.4307 - val_loss: 1.7143 - val_acc: 0.4274
Epoch 40/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6866 - acc: 0.4300 - val_loss: 1.7052 - val_acc: 0.4226
Epoch 41/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6880 - acc: 0.4322 - val_loss: 1.7299 - val_acc: 0.4137
Epoch 42/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6847 - acc: 0.4322 - val_loss: 1.8020 - val_acc: 0.4008
Epoch 43/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6805 - acc: 0.4301 - val_loss: 1.7149 - val_acc: 0.4175
Epoch 44/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6840 - acc: 0.4325 - val_loss: 1.7346 - val_acc: 0.4158
Epoch 45/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6876 - acc: 0.4340 - val_loss: 1.8701 - val_acc: 0.3909
Epoch 46/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6890 - acc: 0.4315 - val_loss: 1.6840 - val_acc: 0.4277
Epoch 47/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6870 - acc: 0.4334 - val_loss: 1.7338 - val_acc: 0.4219
Epoch 48/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6895 - acc: 0.4318 - val_loss: 1.7242 - val_acc: 0.4246
Epoch 49/50
50000/50000 [==============================] - 148s 3ms/step - loss: 1.6877 - acc: 0.4366 - val_loss: 1.8392 - val_acc: 0.4001
Epoch 50/50
50000/50000 [==============================] - 149s 3ms/step - loss: 1.6844 - acc: 0.4347 - val_loss: 1.7266 - val_acc: 0.4192
Model Saved

There are still ways that I can obviously improve my model and currently I am working on data augmentation to improve my accuracy while also dealing with some over-fitting issues.

But I just wanted to know in general what are some aspects of creating a cnn model should one keep in mind? As well as tips/tricks for creating improvement in results? Also if you have any advice or critiques on my current model any input would be appreciated (i'm still working on improving it) -- thank you

Improve this question
$\endgroup$
1
$\begingroup$

My suggestion is to change the model from classification to a regression model. Age is a continuous variable, therefore it is more logical to frame this as a regression problem.

Improve this answer
$\endgroup$
4
  • $\begingroup$ That's actually what I was intending to do next. Is there any simple way to convert this CNN to regression based? Or is a regression based cnn different in regards to layering? $\endgroup$ – BearsBeetBattlestar Dec 16 '18 at 16:03
  • $\begingroup$ It is really easy. You need to change your labels and your model. Take your model and remove the non-linear activation of the last layer (softmax) and change the output to be just a single neuron (without any non-linear activation). Then change your labels from one-hot-encoding to simple numbers. You are now ready to start training your regression model. $\endgroup$ – Mark.F Dec 16 '18 at 16:14
  • $\begingroup$ really its that simple? I assumed we would also have to change the loss in compile. Also in classification the output for example in my case would be 10 values each representing the probability (in total of 1) for each of the 10 cases what would the 1 output in a regression model represent? $\endgroup$ – BearsBeetBattlestar Dec 16 '18 at 16:57
  • $\begingroup$ Sorry, forgat about that one. You also need to change the loss to something like MSE. And the output is simply the age in your case. No other meaning $\endgroup$ – Mark.F Dec 16 '18 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.