0
$\begingroup$

I want to train a range of models, that are very similar; the convoluted layers are the same but the flattened part changes. Example; different density, initially I have model.add(Dense(512)) but I would also like to test 1000 and 4096; I would also like to test adding another layer.

I have no problem creating this models and running them, but it feels like a waste of time to retrain the convoluted layers every time when I'm not modifying anything in them.

Is it possible to save the training done in the convoluted layers and reuse that when training the flattened layers?

This is the base model I have:

model = Sequential()

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(32, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2), padding='same'))

model.add(Flatten())
model.add(Dropout(0.2))
# -------------------- Start of Variable Section -------------- 
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
# -------------------- End of Variable Section ----------------
model.add(Dense(1))
model.add(Activation('sigmoid'))

Other options I want to try (changes to be made inside the variable section) are:

model.add(Dense(1000, activation='relu'))
model.add(Dropout(0.2))

or

model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))

etc.

To achieve that, where do I save the model? If I remove the flattened section:

model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))

The model accuracy is unsurprisingly ~none, if I leave the section that is not variable, like this:

model.add(Flatten())
model.add(Dropout(0.2))
model.add(Dense(1))
model.add(Activation('sigmoid'))

I can save the model and load it, but how would I add those flattened layers?

$\endgroup$
0
$\begingroup$

You can simply train your convolutional layers, save the model and load weights from specific layers using the get_weights and set_weights methods (see also this previous answer). After loading the weights for you convolutional layers you can freeze those layers using the trainable attribute to make sure the weights are not changed during training.

$\endgroup$
2
  • $\begingroup$ updated the question; I have seen the answer you pointed me to, but that is not my problem $\endgroup$ – Carlos P Ceballos Dec 31 '20 at 0:47
  • $\begingroup$ Not sure I fully understand your question, simply save the model using model.save() after you have trained your model (or at least passed one input through it to initialize the parameters). You save the whole model (i.e. the model variables in your example), not just a few layers. $\endgroup$ – Oxbowerce Dec 31 '20 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.