11
$\begingroup$

I'm working on Seq2Seq model using LSTM from Keras (using Theano background) and I would like to parallelize the processes, because even few MBs of data need several hours for training.

It is clear that GPUs are far much better in parallelization than CPUs. At the moment, I only have CPUs to work with. I could access 16 CPUs(2 Threads per core X 4 cores per socket X 2 sockets)

From the doc of multi-core support in Theano, I managed to use all the four cores of a single socket. So, basically the CPU is at 400% usage with 4CPUs used and the remaining 12 CPUs remain unused. How do I make use of them too. Tensorflow could also be used instead of Theano background, if it works.

enter image description here

$\endgroup$
7
$\begingroup$

In order to set the number of threads used in Theano (and, therefore, the number of CPU cores), you'll need to set a few parameters in the environment:

import os
os.environ['MKL_NUM_THREADS'] = '16'
os.environ['GOTO_NUM_THREADS'] = '16'
os.environ['OMP_NUM_THREADS'] = '16'
os.eviron['openmp'] = 'True'

This should allow you to use all cores of all CPUs.

This can, of course, also be done in Tensorflow:

import tensorflow as tf
from keras.backend import tensorflow_backend as K

with tf.Session(config=tf.ConfigProto(
                    intra_op_parallelism_threads=16)) as sess:
    K.set_session(sess)
    <Your Keras code>
| improve this answer | |
$\endgroup$
  • $\begingroup$ I would be thankful for this code for setting number of cores in R (I am using keras in R and it is using only part of cores ...). Especially for tensorflow. Thanks, Milan $\endgroup$ – gutompf Jan 22 '18 at 17:22
  • 2
    $\begingroup$ You can use Sys.setenv() to set environment variables in R, similar to using os.environ[var] in Python. $\endgroup$ – Thomas Cleberg Jan 22 '18 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.