I have the next code that I am trying to run in parallel:

inputs = range(10) 

def processInput(i,x):
   return model(x)

results = Parallel(n_jobs=num_cores)(delayed(processInput)(i,x) for i in inputs)

Where model is a convolutional neural net, and x a batch of images.

It raises a gigantic error which ends with "PicklingError: Could not pickle the task to send it to the workers."

I just want to run the model in parallel N times with dropout to do some variance calculation later. Am I doing it wrong?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.