2
$\begingroup$

Suppose I have these two models (model1 and model2) trained from same structured data, but different datasets:

# create and fit the LSTM network on dataset1
model = Sequential()
model.add(LSTM(4, input_shape=(look_back, 6)))
model.add(Dense(1))
optimizer = Adam(lr=0.001, decay=0.00001)
model.compile(loss='mean_squared_error', optimizer=optimizer)
model.summary()
model1 = model.fit(trainX, trainY, epochs=100, batch_size=64, verbose=2)
model1.save_weights("Model1.h5")

# create and fit the LSTM network on dataset2
model = Sequential()
model.add(LSTM(4, input_shape=(look_back, 6)))
model.add(Dense(1))
optimizer = Adam(lr=0.001, decay=0.00001)
model.compile(loss='mean_squared_error', optimizer=optimizer)
model.summary()
model2 = model.fit(trainX, trainY, epochs=100, batch_size=64, verbose=2)
model2.save_weights("Model2.h5")

How do I combine Model1.h5 and Model2.h5 to make suppose Model3.h5 which has all the attributes of Model1.h5 and Model2.h5 ?

Any help will be highly appreciated.

Thanks,

$\endgroup$
6
  • $\begingroup$ How different are the datasets ? $\endgroup$ Aug 23 '18 at 9:47
  • $\begingroup$ @alexandre_d - just new input, other than that the structure is the same, as in same column names, same number of columns. $\endgroup$
    – Jazz
    Aug 23 '18 at 9:48
  • $\begingroup$ Does it represent the same distribution of data/tasks ? $\endgroup$ Aug 23 '18 at 9:51
  • $\begingroup$ @alexandre_d- absolutely. $\endgroup$
    – Jazz
    Aug 23 '18 at 9:51
  • $\begingroup$ Just make predictions using both models and take the mean :-) $\endgroup$
    – n1k31t4
    Aug 23 '18 at 11:20
1
$\begingroup$

Merging models is not as simple as it seems. In my opinion, I see multiple possibilities :

  1. Don't merge models, merge datasets and retrain : this is in my opinion the most reliable solution, models are fitted from a dataset which represent a certain distribution of data and features. If you can retrain : retrain, especially if datasets are differents.

  2. Create a model over models : use your models as features extractors (cut final part) and build a siamese network (which compute features in parallel from two submodels), merge features obtained and add top layer which will classify from these new set of features (just retrain final part, freeze your models's layers). With this approach, you can retrain a new model which will keep both models's logic without having to retrain a full network. This approach is better than the first if retrain a model from scratch is too constraining.

  3. Play around models's weights : you can access to weights of models and create a third by taking mean of weigths's layers. In my opinion, you can try this but I highly doubt of this approach, instead take the best model and use it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.