2
$\begingroup$

I have a function I am trying to debug which is yielding the following error message:

ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 128, 80, 256), (None, 64, 80, 80)]

I'm running a kernel from a Kaggle competition called Dstl Satellite Imagery Feature Detection (kernel available here)

This is the function where I am having issues merging a list of tensors into a single tensor:

 def get_unet():
    inputs = Input((8, ISZ, ISZ))
    conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(inputs)
    conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv1)
    pool1 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv1)

    conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool1)
    conv2 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv2)
    pool2 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv2)

    conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool2)
    conv3 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv3)
    pool3 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv3)

    conv4 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool3)
    conv4 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv4)
    pool4 = MaxPooling2D(pool_size=(2, 2), dim_ordering="th")(conv4)

    conv5 = Convolution2D(512, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(pool4)
    conv5 = Convolution2D(512, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv5)

    up6 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv5), conv4], mode='concat', concat_axis=1)
    conv6 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up6)
    conv6 = Convolution2D(256, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv6)

    up7 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv6), conv3], mode='concat', concat_axis=1)
    conv7 = Convolution2D(128, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up7)
    conv7 = Convolution2D(128, 3, 3, activation='relu', border_mode='same')(conv7)

    up8 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv7), conv2], mode='concat', concat_axis=1)
    conv8 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up8)
    conv8 = Convolution2D(64, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv8)

    up9 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv8), conv1], mode='concat', concat_axis=1)
    conv9 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(up9)
    conv9 = Convolution2D(32, 3, 3, activation='relu', border_mode='same', dim_ordering="th")(conv9)

    conv10 = Convolution2D(N_Cls, 1, 1, activation='sigmoid', dim_ordering="th")(conv9)

    model = Model(input=inputs, output=conv10)
    model.compile(optimizer=Adam(), loss='binary_crossentropy', metrics=[jaccard_coef, jaccard_coef_int, 'accuracy'])
    return model

I'm running keras with a TensorFlow backend. My thoughts are that there are some compatibility issues with versions of software (i.e. the original code is over a year old). Or, perhaps I need to reshape the data somehow.

What might be causing this error?

This is the full error:

Traceback (most recent call last):

  File "<ipython-input-1-e8f13915ac9b>", line 1, in <module>
    runfile('/Users/aaron/temp/tmp/kaggle_dstl_v3.py', wdir='/Users/aaron/temp/tmp')

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 513, in <module>
    model = train_net()

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 416, in train_net
    model = get_unet()

  File "/Users/aaron/temp/tmp/kaggle_dstl_v3.py", line 294, in get_unet
    up8 = merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv7), conv2], mode='concat', concat_axis=1)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 458, in merge
    name=name)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 111, in __init__
    node_indices, tensor_indices)

  File "/Users/aaron/anaconda3/envs/kaggle-dstl-env/lib/python3.6/site-packages/keras/legacy/layers.py", line 191, in _arguments_validation
    'Layer shapes: %s' % (input_shapes))

ValueError: "concat" mode can only merge layers with matching output shapes except for the concat axis. Layer shapes: [(None, 128, 80, 256), (None, 64, 80, 80)]
$\endgroup$
2
$\begingroup$

First of all, you are correct that your code is old as some functions being used are deprecated (e.g. Convolution2D is now Conv2D see here).

However, the error clearly states that you are trying to concatenate two tensors that their dimensions do not match. When concatenating two tensors along a specific axis, all other dimensions except the one being concatenated must be the same. In your case, you are trying to concatenate along axis=1, but the last dimension is different (256 for the first tensor and 80 for the second).

I would recommend basing your U-Net code on a newer implementation (link1, link2, link3).

These implementations replace your

merge([UpSampling2D(size=(2, 2), dim_ordering="th")(conv5), conv4], mode='concat', concat_axis=1)

layers with

up_conv5 = UpSampling2D(size=(2, 2), data_format="channels_last")(conv5)
ch, cw = get_crop_shape(conv4, up_conv5)
crop_conv4 = Cropping2D(cropping=(ch,cw), data_format="channels_last")(conv4)
up6   = concatenate([up_conv5, crop_conv4], axis=concat_axis)`

which also crops the height and the width of the previous layer to perform the concatenation correctly (the shape of the crop is determined through an auxiliary function).

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.