I created two convolutional neural networks (CNN), and I want to make these networks work in parallel. Each network takes different type of images and they join in the last fully connected layer.
How to do this?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this communityYou essentially need a multi-input model. This can only be done through keras' functional api and can work with the pretrained nets in keras.applications
. To create one you can do this:
from keras.layers import Input, Conv2D, Dense, concatenate
from keras.models import Model
1) Define your first model:
in1 = Input(...) # input of the first model
x = Conv2D(...)(in1)
# rest of the model
out1 = Dense(...)(x) # output for the first model
2) Define your second model:
in2 = Input(...) # input of the first model
x = Conv2D(...)(in2)
# rest of the model
out2 = Dense(...)(x) # output for the first model
3) Merge the two models and conclude the network:
x = concatenate([out1, out2]) # merge the outputs of the two models
out = Dense(...)(x) # final layer of the network
4) Create the Model:
model = Model(inputs=[in1, in2], outputs=[out])