n_jobs=-1 or n_jobs=1?

Question

I am confused regarding the n_jobs parameter used in some models and for CV. I know it is used for parallel computing, where it includes the number of processors specified in n_jobs parameter. So if I set the value as -1, it will include all the cores and their threads for faster computation. But this article:-
https://machinelearningmastery.com/multi-core-machine-learning-in-python/#comment-617976

states that using all cores for training, evaluation and hyperparameter tuning is a bad idea. The crux of the article is as follows:-

1.)When using k-fold cross-validation, it is probably better to assign cores to the resampling procedure and leave model training single core.

2.)When using hyperparamter tuning, it is probably better to make the search multi-core and leave the model training and evaluation single core.

But common sense says that setting n_jobs = -1 everywhere will include all cores for faster computation and hence result in less run time. Can anyone clarify?

Answer 1

This is about distributed computing: let's say that you have 100 tasks and 10 cores available. You parallelize your tasks so that each core processes 10 of them. Now let's imagine that the task involves some subtasks and internally tries to use all the cores available: at the two levels of parallelization the processes compete for the cores, causing a loss in performance because there are more pending processes than available cores. This can usually be observed with some utilities to visualize the activity of the cores.

This is why it's much more efficient to control at which level parallelization should happen, in order to minimize competition between processes. In my example in theory the top tasks could be distributed say into 5 processes while each task could still use 2 cores. However it's usually simpler and more efficient to just decide one level of parallelization: in your case either the training or the parallelization, but not both. Imho the choice is not obvious though, in particular it depends on how intensive the training process is.