0
$\begingroup$

I'm trying to optimize the hyperparameters of my model using RandomizedSearchCV. However, it doesn't stop running even if I define few iterations. Someone could help me? The code I'm using is presented below:

def build_classifier(optimizer, units, alpha, l1):
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.LSTM(units, kernel_regularizer = regularizers.l1(l1 = l1), input_shape= (None, n_features), return_sequences = True))
    model.add(tf.keras.layers.LSTM(units, kernel_regularizer = regularizers.l1(l1 = l1), return_sequences = True))
    model.add(tf.keras.layers.LSTM(units, kernel_regularizer = regularizers.l1(l1 = l1), return_sequences = False))
    model.add(tf.keras.layers.Dense(5))

    model.compile(optimizer = optimizer, loss = 'mae')
    return model

from keras.wrappers.scikit_learn import KerasRegressor
parameters ={'optimizer':['adam','rmsprop','SGD'],
            'units':[4,8,16,32,64,128],
            'alpha':[0.1,0.2,0.3,0.4,0.5],
            'l1':[0.00001,0.0001,0.001,0.01,0.1],
            'epochs':[2,5]}

classifier = KerasRegressor(build_fn = build_classifier)

from sklearn.model_selection import RandomizedSearchCV
random_search = RandomizedSearchCV(estimator = classifier, param_distributions = parameters, n_iter = 1, n_jobs = 1, cv = 5, scoring = 'neg_mean_absolute_error')

y_hyperparameters = np.reshape(y_hyperparameters, (y_hyperparameters.shape[0], n_features))

random_search.fit(X_hyperparameters, y_hyperparameters, verbose = 1, batch_size = 1, validation_data = (test_X_hyperparameters, test_y_hyperparameters), shuffle = False)

print('Random Best score:', random_search.best_score_)
print('Random Best params:', random_search.best_params_)
$\endgroup$
3
  • $\begingroup$ What do you mean by 'doesn't stop running'? I suspect that the cross validation is still happening but simply takes a long time because of a combination of the type of model, total number of hyperparameters to check, and the number of folds in your cross-validation. $\endgroup$
    – Oxbowerce
    Commented Jul 30, 2021 at 14:08
  • $\begingroup$ When I say it doesn't stop running, I mean that assuming I have 5 epochs. It runs all 5 epochs, then runs 5 epochs again and continues. This behavior occurs even when I have only one iteration. $\endgroup$
    – Fernanda
    Commented Jul 30, 2021 at 16:04
  • $\begingroup$ You've specified cv=5, and left refit=True, so even with n_iter=1 you will have 6 fits total. So again, "What do you mean by 'doesn't stop running'?" $\endgroup$
    – Ben Reiniger
    Commented May 6, 2022 at 20:21

1 Answer 1

-1
$\begingroup$

One possible cause of errors is the mixing and matching of completely different algorithms: regressors, classifiers, and sequence models (i.e., LSTM). You import a regressor, you name it a classifier, and the actual model is an LSTM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.