0
$\begingroup$

I have a problem. I have trained a CNN model for an NLP classification problem and combined it with other features. I am using Concatenate to concatenate the two layers with it.

My question is how does this chaining work? How can this be thought of?

class CNN_1D:
    def __init__(self, x, y):
        self.x = x
        self.y = y

    def forward(self):
            # filter_sizes = [1,2,3,5]
            # num_filters = 32
            extra_nb_features = df_train.shape[1]

            inp = Input(shape=(maxlen, ))
            extra_inp = Input(shape=(extra_nb_features, ))

            x = Embedding(embedding_matrix.shape[0], 300, weights=[embedding_matrix], trainable=False)(inp)
            x = SpatialDropout1D(0.4)(x)

            x = Conv1D(256, 7, activation='relu')(x)
            x = MaxPooling1D()(x)

            x = Dropout(0.2)(x)  
            x = Flatten()(x)
            combined = Concatenate(axis=-1)([x, extra_inp])

            combined = Dropout(0.15)(combined)

            outp = Dense(numbmer, activation="softmax")(combined)

            model = Model(inputs=[inp, extra_inp] , outputs=outp)
            model.summary()
            return model
$\endgroup$

1 Answer 1

1
$\begingroup$

Conceptually the first input inp is embedded and passed through all the layers that have x as an output. extra_inp is a set of raw inputs that are passed in with the convoluted inp to the Concatenate layer to return a single set of inputs (a tensor) that is going to be used for the final prediction. Concatenate merges both inputs, the ones that are “preprocessed” and the unaltered ones. You can check more details in the keras documentation

Here's a toy model that can be used as en example:

from tensorflow.keras.layers import Input, Conv1D, Flatten, Concatenate, Dense
from tensorflow.keras import Model

inputs = Input(shape= (32,1))
h = Conv1D(filters= 16, kernel_size= 5, activation= 'relu', padding= 'same')(inputs)
h = Flatten()(h)
aux_inputs = Input(shape= (12,))
h = Concatenate()([h, aux_inputs])
output = Dense(20, activation= 'sigmoid')(h)

model = Model(inputs= [inputs, aux_inputs], outputs= output)

The summary of the model:

multiple input model

The convolution layer takes a batch of 32-dimensional vectors and returns a tensor with shape batch_size, new_input_dimension, filters. In this case, the model is configured to have padding = 'same' so the dimensions remain the same. The output of the Conv1d layer is then unrolled by a Flatten layer for it to have the right shape to concatenate it with the extra input (which is also one dimensional). Finally, this extended tensor is passed in to the final Dense layer to return the output of the model.

The following graph is quite straightforward:

enter image description here

Hope you find it helpful!

$\endgroup$
2
  • $\begingroup$ Could you give an example of how the concatenation works? $\endgroup$
    – Test
    Oct 24, 2022 at 8:27
  • $\begingroup$ yup, just added an example to the answer $\endgroup$
    – jees
    Oct 24, 2022 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.