0
$\begingroup$

I'm training a sample LTSM with keras / scikit-learn and I want to use the KerasClassifier wrapper, but it is giving me very odd results comparying it to a plain model.fit:

model = build_model()
history = model.fit(x_train, y_train, epochs=10, batch_size=50)
Epoch 1/10
89/89 [==============================] - 5s 52ms/step - loss: 0.0105 - mean_absolute_error: 0.0105
Epoch 2/10
89/89 [==============================] - 4s 48ms/step - loss: 0.0054 - mean_absolute_error: 0.0054
Epoch 3/10
89/89 [==============================] - 4s 49ms/step - loss: 0.0048 - mean_absolute_error: 0.0048
Epoch 4/10
89/89 [==============================] - 4s 49ms/step - loss: 0.0052 - mean_absolute_error: 0.0052
Epoch 5/10
89/89 [==============================] - 5s 51ms/step - loss: 0.0060 - mean_absolute_error: 0.0060
Epoch 6/10
89/89 [==============================] - 4s 49ms/step - loss: 0.0048 - mean_absolute_error: 0.0048
Epoch 7/10
89/89 [==============================] - 4s 50ms/step - loss: 0.0043 - mean_absolute_error: 0.0043
Epoch 8/10
89/89 [==============================] - 5s 52ms/step - loss: 0.0044 - mean_absolute_error: 0.0044
Epoch 9/10
89/89 [==============================] - 5s 52ms/step - loss: 0.0041 - mean_absolute_error: 0.0041
Epoch 10/10
89/89 [==============================] - 5s 51ms/step - loss: 0.0044 - mean_absolute_error: 0.0044

You see here a very low MAE, but when doing something similar with the wrapper:

num_epochs = 10
num_batch_size = 50
estimator = KerasClassifier(build_fn=build_model,
                            epochs=num_epochs,
                            batch_size=num_batch_size,
                            verbose=0)
estimator.fit(x_train, y_train, epochs=10, batch_size=50, verbose=1)

Epoch 1/10
89/89 [==============================] - 5s 52ms/step - loss: 1765.9719 - mean_absolute_error: 1765.9719
Epoch 2/10
89/89 [==============================] - 5s 51ms/step - loss: 1758.8699 - mean_absolute_error: 1758.8699
Epoch 3/10
89/89 [==============================] - 4s 50ms/step - loss: 1753.9438 - mean_absolute_error: 1753.9438
Epoch 4/10
89/89 [==============================] - 4s 49ms/step - loss: 1749.2325 - mean_absolute_error: 1749.2325
Epoch 5/10
89/89 [==============================] - 4s 48ms/step - loss: 1744.6199 - mean_absolute_error: 1744.6199
Epoch 6/10
89/89 [==============================] - 4s 49ms/step - loss: 1740.0754 - mean_absolute_error: 1740.0754
Epoch 7/10
89/89 [==============================] - 4s 48ms/step - loss: 1735.5726 - mean_absolute_error: 1735.5726
Epoch 8/10
89/89 [==============================] - 4s 49ms/step - loss: 1731.1071 - mean_absolute_error: 1731.1071
Epoch 9/10
89/89 [==============================] - 4s 48ms/step - loss: 1726.6743 - mean_absolute_error: 1726.6743
Epoch 10/10
89/89 [==============================] - 5s 53ms/step - loss: 1722.2772 - mean_absolute_error: 1722.2772

Exact same data and model. Is there anything obvious I'm missing here?

scikit-learn==0.23.1
tensorflow==2.2.0

Build model function:

def build_model():
    # build the LSTM model
    model = Sequential()
    model.add(LSTM(50, return_sequences=True, input_shape=(x_train.shape[1], 1)))
    model.add(Dropout(0.1))
    model.add(LSTM(50, return_sequences=True))
    model.add(Dropout(0.1))
    model.add(LSTM(50))
    model.add(Dense(1))

    # compile the model
    model.compile(optimizer='adam', loss='mean_absolute_error', metrics=['mean_absolute_error'])
    return model
$\endgroup$
  • $\begingroup$ Seeing your loss function, it seems you should use - KerasRegressor (tf.keras.wrappers.scikit_learn.KerasRegressor) $\endgroup$ – 10xAI May 31 at 5:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.