enter image description here

I have this below architecture I would like to understand below question This is my code and my image size is 50X50

 model.add(Conv2D(16, (3,3), input_shape=(image_x, image_y, 1), activation='relu'))
        model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
        model.add(Conv2D(32, (3,3), activation='relu'))
        model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
        model.add(Conv2D(64, (3,3), activation='relu'))
        model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))
        model.add(Dense(128, activation='relu'))
        model.add(Dense(num_of_classes, activation='softmax'))`
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

My question are : 1] dense_1 : Flatten has 1600 input and out will be 128 so how it will select 128 out put from 1600 input? 2] In which layer lost function and optimizor will be apply.

  1. You don't select 128 from 1600. Dense layer performs a matrix-vector multiplication. First, flatten layer basically flat the output of your max_pool layer(5*5*64) to a vector (1600*1) (note that 5*5*64 =1600). It is just a re-arrangment of the element. We flatten max-pool layer because we want to use the output of max-pool layer as the input of dense layer (dense layer is also called fully connected layer or multiple layer perceptron). Dense layer is nothing but a vector-matrix multiplication. Say you have an input vector(I) whose shape is 1600*1, and you want your output (O) shape to be 128*1. Then your dense layer will perform the following operation: O=WI, where W is a 128*1600 weight matrix. That is how you got 128 output.The dimension of the W matrix is specified by your input shape and output shape and thus, after you specify the input/output shape, the W matrix is implicitly auto-generated by tensorflow. Btw, your "relu" function in the denselayer block, which is called non-activation function, performs relu function calculate on each element of your output vector (128*1) so it doesn't affect your final output shape.

  2. Normally, all layers will be optimized. The loss function defines a loss between your true label and the output of your neural network (which is the last dense layer in your case). The gradient of the loss back propagates to each layer through the back propagation algorithm(you can check it online, but basically it is just chain rule). Thus, each layer has its own error gradients. We then use these error gradients to update coefficients of each layer to make the overall loss smaller (the loss defined by your loss function). The operation is called gradient descent algorithm. Since each layer updates itself separately, you can always freeze some layers so that these coefficients don't update, which can be done in tensorflow.

  • $\begingroup$ you are welcome $\endgroup$ – Tong Lin Apr 17 '19 at 3:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.