2
$\begingroup$

I am using a CNN (adapted from a few links on the net) for an image classification task. There are about 8000 images of size 128x128 each. They are of 13 different classes. Following is output of model.summary()):

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
batch_normalization_1 (Batch (None, 128, 128, 3)       12        
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 128, 128, 32)      896       
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 64, 64, 32)        0         
_________________________________________________________________
batch_normalization_2 (Batch (None, 64, 64, 32)        128       
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 64, 64, 64)        18496     
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 32, 32, 64)        0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 32, 32, 64)        256       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 32, 32, 128)       73856     
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 16, 16, 128)       0         
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 16, 16, 64)        73792     
_________________________________________________________________
global_average_pooling2d_1 ( (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 13)                845       
=================================================================
Total params: 168,793
Trainable params: 168,339
Non-trainable params: 454

How does one analyze this model summary and how can this model be improved?

$\endgroup$
6
  • $\begingroup$ conv2d_4 (Conv2D) (None, 16, 16, 64) 73792 _________________________________________________________________ global_average_pooling2d_1 ( (None, 64) 0 _________________________________________________________________ dense_1 (Dense) (None, 13) 845 ================================================================= Avoid using that pooling layer, it lessens the information while sending them to dense layer. Also try to add another dense layer for better accuracy. Your extracted features may . $\endgroup$ Oct 6, 2018 at 10:22
  • $\begingroup$ not be linearly separable $\endgroup$ Oct 6, 2018 at 10:22
  • $\begingroup$ Dense layer should be added at end (in the part you mentioned in your comment) or can it be added in previous layers also? $\endgroup$
    – rnso
    Oct 6, 2018 at 10:56
  • $\begingroup$ Also, should max_pooling and batch_normalization be added with every convolution2d layer? $\endgroup$
    – rnso
    Oct 6, 2018 at 10:59
  • $\begingroup$ The simple answer is you need to experiment and gain experience... Asking won't help we don't have that data source Also you don't analyze summary, rather the weights of in between Convs let's say via Visualisation Also please update your Question with your Confusion Matrix maybe.... $\endgroup$
    – Aditya
    Oct 6, 2018 at 11:20

1 Answer 1

1
$\begingroup$

What is the meaning of: Non-trainable params: 454? Should this ideally be 0? If so, how can this be made 0.

I guess it is due to using batch normalisation. Try to ascertain by investigating the math of that. If you see the paper, you can easily find out that the bias terms and $\beta$, if I remember, both get added. Consequently, one can be ignored and won't be trained because it is not needed.

Dense layer should be added at the end (in the part you mentioned in your comment) or can it be added in previous layers also?

Not really, dense layers should be employed after conv layers. What they do is classifying the extracted features obtained by conv layers. About conv layers, they are employed for reducing the number of parameters and finding local patterns.

There is no consensus on how to change the number of filters in convolutional layers, at least as far as I know. But there is a point here. In the following lines of your code, you've employed a kind of pooling layer just before dense layer. If the number of activations coming from conv layer is many, you can use it but consider that by doing so, you ignore important features. I suggest you not doing that, especially, for the last conv layer. Also, try to increase the number of neurons in a dense layer or add extra layers for better accuracy.

conv2d_4 (Conv2D)            (None, 16, 16, 64)        73792     
_________________________________________________________________
global_average_pooling2d_1 ( (None, 64)                0         
_________________________________________________________________
dense_1 (Dense)              (None, 13)                845       
=================================================================

About using batch norm, it is used due to a kind of problem which is called Covariat Shift. It simply tries to keep the distribution of the outputs of different layers in order to facilitate the learning process.

Based on your questions, I highly recommend you watching professor Andrew Ng's course about ConvNets in Coursera.

$\endgroup$
5
  • $\begingroup$ Thanks for a good answer. Should batchnorm-conv2d-maxpooling set always used or are there any alternatives? $\endgroup$
    – rnso
    Oct 6, 2018 at 14:35
  • $\begingroup$ If you have a very deep network, employ batch norm. max pooling is for decreasing the number of parameters and adding partial spatial invariance to the network. $\endgroup$ Oct 6, 2018 at 15:19
  • $\begingroup$ Last point: any role of Flatten or Dropout layers in this situation? $\endgroup$
    – rnso
    Oct 6, 2018 at 15:46
  • $\begingroup$ Flatten should be used for connecting conv to dense. Dropout is used for avoiding overfitting. $\endgroup$ Oct 6, 2018 at 15:59
  • $\begingroup$ Believe me! Take a course, otherwise you will be completely blind. There are abundant hyper parameters. $\endgroup$ Oct 6, 2018 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.