0
$\begingroup$

I would like to train a model with Keras and TensorFlow. My input consists of images and some additional features. I would like to use conv2d for the images and dense for the other inputs.

The question is how do I build an architecure that has multiple input types where each one gets connected to a different layer?

$\endgroup$

1 Answer 1

1
$\begingroup$

This is quite easy to do using the keras functional API. Assuming you have an image of size 28 by 28 and 5 additional features, your model could look something like this:

from tensorflow.keras import Model, Input
from tensorflow.keras.layers import Conv2D, MaxPool2D, Dense, Flatten, concatenate

input_image = Input(shape=(28, 28, 3))
input_features = Input(shape=(5,))

# apply convolutional layers to image branch
x = Conv2D(32, 3)(input_image)
x = Conv2D(32, 3)(x)
x = MaxPool2D(2)(x)
x = Flatten()(x)

# concatenate flattened image branch with input features
concat = concatenate([x, input_features])

# apply dense layers on concatenated data
dense = Dense(64)(concat)
output = Dense(64)(dense)

# create models using inputs and output specified above
model = Model(inputs=[input_image, input_features], outputs=output)

When visualized the model structure would like this:

from tensorflow import keras

keras.utils.plot_model(model, show_shapes=True)

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.