2
$\begingroup$

I have recently started learning deep learning. In machine learning using sklearn library with n_jobs = -1 all my cpu cores are used and this speeds the grid search. Now I am trying to fit an rnn model on training data, which is taking a lot of time. Is there a way I can speed up the training?

# Initialising the RNN
regressor = Sequential()

# Adding the first LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True, input_shape = (X_train.shape[1], 7)))
regressor.add(Dropout(0.2))

# Adding a second LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding a third LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50, return_sequences = True))
regressor.add(Dropout(0.2))

# Adding a fourth LSTM layer and some Dropout regularisation
regressor.add(LSTM(units = 50))
regressor.add(Dropout(0.2))

# Adding the output layer
regressor.add(Dense(units = 1))

# Compiling the RNN
regressor.compile(optimizer = 'adam', loss = 'mean_squared_error')

# Fitting the RNN to the Training set
regressor.fit(X_train, y_train, epochs = 100, batch_size = 32,shuffle=False)
$\endgroup$
1
  • $\begingroup$ By default it will pick it, you training is slow because of your hardware. $\endgroup$
    – Aditya
    Commented Mar 10, 2019 at 2:55

1 Answer 1

2
$\begingroup$

Here you do a single fit() of the model whose name tells for itself - Sequential.

Unless you are doing cross-validation or some kind of distributed learning with multiple models, there is no benefit of running several fits in parallel.

However, you can have significant speed-up on iteration level, depending on how Keras backend is configured to work with your hardware.

Assuming you are using Keras with the tensorflow backend (default):

  • If you are running on CPU, tensorflow should already pick up all available cores by default (if not, it might help checking How to run Keras on multiple cores?);

  • If you are running on GPU:

    (1) make sure Keras backend can see your GPU

    from keras import backend
    print(backend.tensorflow_backend._get_available_gpus())
    

    If your GPU is not showing up here, some configuration might be necessary on your system (it deserves another topic);

    (2) take a look at the CuDNNLSTM - it is the Keras LSTM implementation optimized for GPU.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.