# How do I provide input and output for such a network structure in keras

I am trying to create a CNN network for classification purposes, the network with both input and output is illustrated as such:

The input the image is separated into sections, each section is given to a certain model in which a convolution and pooling is applied. This is done as such to ensure full weight share does not occur, as I am not interested in the complete image shares the same weight. After the convolution an pooling, is all the output of models concatenated into one, and fed into a fully connected layer, which then performs classification and outputs a class.

The input I have is the image (either the complete or separated into sections) and the output being the class, but I am not sure how I should train the model, given these input and output.

The way I've constructed the model is as such:

#Convolution for different section conv2d_[convolution#]_[model#]
conv2d_1_1 = Conv2D(filters = 32, kernel_size = (3,3) , padding = "same" , activation = 'relu' , input_shape = (3,6,3))
conv2d_2_1 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_1_1)
conv2d_3_1 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_2_1)
conv2d_4_1 = Conv2D(filters = 32, kernel_size = (1,1) , padding = "same" , activation = 'relu' )(conv2d_3_1)
conv2d_4_1_flatten = Flatten()(conv2d_4_1)

conv2d_1_2 = Conv2D(filters = 32, kernel_size = (3,3) , padding = "same" , activation = 'relu' , input_shape = (3,6,3))
conv2d_2_2 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_1_2)
conv2d_3_2 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_2_2)
conv2d_4_2 = Conv2D(filters = 32, kernel_size = (1,1) , padding = "same" , activation = 'relu' )(conv2d_3_2)
conv2d_4_2_flatten = Flatten()(conv2d_4_2)

conv2d_1_3 = Conv2D(filters = 32, kernel_size = (3,3) , padding = "same" , activation = 'relu' , input_shape = (3,6,3))
conv2d_2_3 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_1_3)
conv2d_3_3 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_2_3)
conv2d_4_3 = Conv2D(filters = 32, kernel_size = (1,1) , padding = "same" , activation = 'relu' )(conv2d_3_3)
conv2d_4_3_flatten = Flatten()(conv2d_4_3)

conv2d_1_4 = Conv2D(filters = 32, kernel_size = (3,3) , padding = "same" , activation = 'relu' , input_shape = (3,6,3))
conv2d_2_4 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_1_4)
conv2d_3_4 = Conv2D(filters = 64, kernel_size = (3,3) , padding = "same" , activation = 'relu' )(conv2d_2_4)
conv2d_4_4 = Conv2D(filters = 32, kernel_size = (1,1) , padding = "same" , activation = 'relu' )(conv2d_3_4)
conv2d_4_4_flatten = Flatten()(conv2d_4_4)

#Merging the output of the convolution
merge = Merged([conv2d_4_1_flatten, conv2d_4_2_flatten,
conv2d_4_3_flatten, conv2d_4_4_flatten] mode = 'concat')

#Connecting the merged layer to the fully connected layer
dense1 = Dense(activation = 'relu')(merge)
dense2 = Dense(activation = 'relu')(dense1)
dense3 = Dense(output = 1 ,activation = 'softmax')(dense3)


This (I guess) CNN model should now be connected, but how do I train it given the input and outputs I have? How do I parse it?

My input data is stored as list of numpy.ndarray. Each row of each ndarray, is the input for each input. How do I parse each row to each input? And the output is a vector of classes. I have in total 145 classes, and output is vector length 3 with the true labels extracted from 4 input channels?

• Standard backpropagation techniques to update the weights should work, since all layers of your neural net are differentiable. What exactly are you confused about? Apr 2 '17 at 22:03
• how do i parse my input/output data to keras model.fit ? Apr 3 '17 at 2:51

You can pass in a list to the input parameter when you create the model.

model = Model(input=[conv2d_1_1, conv2d_1_2, conv2d_1_3, conv2d_1_4], output=dense3)


Make the input x a list of four numpy arrays of shape (3,6,3)

Make the output y an array of one-hot vectors corresponding to your 145 classes.

Then call

model.fit(x, y)


You probably want to change your last layer to have 143 dimensions in the output

num_classes =143
Dense(num_classes, activation='softmax', activation = 'softmax')