How to combine two Deep learning model weights into one

Question

Suppose I have these two models (model1 and model2) trained from same structured data, but different datasets:

# create and fit the LSTM network on dataset1
model = Sequential()
model.add(LSTM(4, input_shape=(look_back, 6)))
model.add(Dense(1))
optimizer = Adam(lr=0.001, decay=0.00001)
model.compile(loss='mean_squared_error', optimizer=optimizer)
model.summary()
model1 = model.fit(trainX, trainY, epochs=100, batch_size=64, verbose=2)
model1.save_weights("Model1.h5")

# create and fit the LSTM network on dataset2
model = Sequential()
model.add(LSTM(4, input_shape=(look_back, 6)))
model.add(Dense(1))
optimizer = Adam(lr=0.001, decay=0.00001)
model.compile(loss='mean_squared_error', optimizer=optimizer)
model.summary()
model2 = model.fit(trainX, trainY, epochs=100, batch_size=64, verbose=2)
model2.save_weights("Model2.h5")

How do I combine Model1.h5 and Model2.h5 to make suppose Model3.h5 which has all the attributes of Model1.h5 and Model2.h5 ?

Any help will be highly appreciated.

Thanks,

Answer 1

Merging models is not as simple as it seems. In my opinion, I see multiple possibilities :

1. Don't merge models, merge datasets and retrain : this is in my opinion the most reliable solution, models are fitted from a dataset which represent a certain distribution of data and features. If you can retrain : retrain, especially if datasets are differents.

2. Create a model over models : use your models as features extractors (cut final part) and build a siamese network (which compute features in parallel from two submodels), merge features obtained and add top layer which will classify from these new set of features (just retrain final part, freeze your models's layers). With this approach, you can retrain a new model which will keep both models's logic without having to retrain a full network. This approach is better than the first if retrain a model from scratch is too constraining.

3. Play around models's weights : you can access to weights of models and create a third by taking mean of weigths's layers. In my opinion, you can try this but I highly doubt of this approach, instead take the best model and use it.