How to get an intution for generating Keras CNN layers for a model

Question

I am going through tutorial for handwritten text recognition. And to do hand written digit recognition the author has constructed a Keras model as follows:

# # Creating CNN model

input_shape = (28,28,1)
number_of_classes = 10

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=input_shape))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))

model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))

model.add(Dropout(0.5))
model.add(Dense(number_of_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),metrics=['accuracy'])

model.summary()

history = model.fit(X_train, y_train,epochs=5, shuffle=True,
                    batch_size = 200,validation_data= (X_test, y_test))


model.save('digit_classifier2.h5')

Source (here)

I am very confused that on how has the author choose these layers. I know how Conv2D works by applying filters to an image, I know what is activation function. In short I have a rough understanding of what each term means.

What I am finding it difficult is how do I know what is happening in each step of this code? For example lets take this python code:

values_List=[11,34,43]
for index, num in enumerate(values_List):
    print(index,num)

1. I know that line 1 initializes a list named values_List
2. Line 2 iterates through this list
3. Line 3 prints output as (index of a number , number)

This python code is easy to understand and debug. But I am confused that if there is any error inside the keras layers. How do I proceed to debug this Keras code ?

Answer 1

Layer Choices : I believe this is an issue that requires domain expertise and experience. Neural network in general is still considered a black-box algorithm. As far as I understand what we try to do is improving model based on previous discovery of what actually works for our cases. Basically as you are more familiar with various model you may understand what works and what doesn't work and you will improve your model based on this. For example, you may want to check a more recent papers such as efficientnet, which is a very light yet powerful image classification model. They combined techniques and ideas that people have proven to work before and we have this amazing model. As for simple models like what you did above you parameter search should suffice. You might also consider a more sophisticated approach is using automl approach such as autokeras.

Debugging : I believe the tools you are looking for is model summary and visualization https://keras.io/visualization/. The first one is simple you simply call model.summary() which I believe you have already used on your code, this will get you a summary of the model and if you already understand how some CNN building block works you should have already get the idea how the shape changes and hence you can judge based on this whether or not it satisfy your expectation. As for your model it is still fairly easy to debug, since it is sequential and I believe the points that I mentioned above are enough.

Answer 2

I suggest you learn what each layer in a CNN does in detail from a tutorial such as Machine Learning Mastery or if you prefer videos Sentdex. Building a CNN for handwritten text recognition is not very different from building a CNN for any other purpose so this would give you a good starting point. After that, you can turn back to this code, (or any other tutorial present online for handwritten text recognition)

And for the errors, it is not very different from any Python code. It tells you on which line the error is and a brief description of it. I suppose it requires some getting used to but if you understand regular Python error messages, you can understand Keras errors. And of course you can always post here the errors you have problems figuring out.