Keras load pre-trained weights. Shape mismatch

Question

I have some trouble loading pre-trained weights with Keras. Let's say I have a keras model model and that my weights are stored at my_weights.h5.

I try to load my weights as follow :

model.load_weights("my_weights.h5", by_name=True)

But this give me the following error :

Layer #1 (named "conv2d_1"), weight <tf.Variable 'conv2d_1/kernel:0' shape=(3, 3, 32, 64) dtype=float32> has shape (3, 3, 32, 64), but the saved weight has shape (32, 3, 3, 3).

So I tried to see what was the shape of my weights and my model structure :

for layer in model_body.layers :
    print(layer.name+" : input ("+str(layer.input_shape)+") output ("+str(layer.output_shape)+")")

print("__")
with h5py.File(weights_filepath, 'r') as f:
    for k in f.keys():
        for l in f[k].keys():
            for m in f[k][l].keys():
                print(k+ " : " + m + " : " + str(f[k][l][m].shape))


conv2d_1 : input ((None, None, None, 32)) output ((None, None, None, 64))
__
conv2d_1 : kernel:0 : (3, 3, 3, 32)

(I kept only the layer that appear in the error)

By seeing this, I don't understand why the shapes mismatch, and where the shape (3, 3, 32, 64) in the error come from). Am I missing something ?

Answer 1

In relation to the issue being related to the trainable attribute https://datascience.stackexchange.com/a/84067/51317 and if it's difficult to figure out which weights were set to trainable, one option is to try loading the weights by name with something like this (this doesn't cover all scenarios):

def load_weights_by_name(model, path, verbose=False):
    import h5py
    def load_model_weights(cmodel, weights):
        for layer in cmodel.layers:
            print(layer.name)
            if hasattr(layer, 'layers'):
                load_model_weights(layer, weights[layer.name])
            else:
                for w in layer.weights:
                    _, name = w.name.split('/')
                    if verbose:
                        print(w.name)
                    try:
                        w.assign(weights[layer.name][name][()])
                    except:
                        w.assign(weights[layer.name][layer.name][name][()])

    with h5py.File(path, 'r') as f:
        load_model_weights(model, f)

Unfortunately, the problem happens because the order of the weights changes when saving the model with modified trainable attributes; however, the function for loading the weights by name in keras does not check the order of weights when it tries to match the weight values with the symbolic weights https://github.com/keras-team/keras/blob/98a762224578cf5e15be39fddf6917cf8efea6e0/keras/saving/hdf5_format.py#L782

Answer 2

I have solved the kind of issue as follows. Hope the solution would be helpful.

1. Delete "by_name=True"

# -model.load_weights(weights_path, by_name=True)
model.load_weights(weights_path)

2. Change the number of class

While it throws another throwing "ValueError: Shapes (1536, 1000) and (1536, 1001) are incompatible", I change num_classes from 1000 to 1001. And then it shows the correct model summary.

# -num_classes = 1000
num_classes = 1001

Answer 3

I had a similar error and was extremely puzzled

ValueError: Shapes (32,) and (3, 3, 32, 64) are incompatible

I eventually figured out that I had modified the trainable attribute of the model. (I was doing transfer learning for the final few layers, and then switching to training the full model.) @Supratim's suggestion of checking the summary was what tipped me off, because that shows the number of trainable params.

The error message is a bit tricky to deciper, because I wouldn't have thought that trainable changes any shapes. I guess the lesson to learn is that the model has to match exactly.