1. what are the rows before the first red row? I thought it may be the combinations of parameters but that doesn't make a lot of sense because those are not enough
Parameters, which are the candidates of the CV are printed
2. What is the meaning of the row between the red rows? Why those parameters are there? is this after one CV?
It is printed after the completion of the 3rd(as defined in your code) fold for a parameter Candidate
3. "Done one task" - what is the task? one CV?
This is from the joblib I believe. It's the completion of one fold i.e. one out of 36.
4. Why do I get a score only for some of the parameters?
The score is printed after the completion of 3(as defined in your code) fold of each parameter candidate
In a multi-thread setup, we can't rely on the sequence of the messages as it will depend on the Thread allocation.
My answer is based on this small setup.
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import RandomizedSearchCV
iris = load_iris()
logistic = LogisticRegression(solver='saga', tol=1e-2, random_state=0)
distributions = dict(C=[0.1,0.5,1.0],
penalty=['l2', 'l1'], max_iter=[50,100])
clf = RandomizedSearchCV(logistic, distributions, random_state=0,n_iter = 5, cv = 2, verbose=10)
from joblib import Parallel, delayed, parallel_backend
with parallel_backend('threading',n_jobs=12):
search = clf.fit(iris.data, iris.target)
Output
Fitting 2 folds for each of 5 candidates, totalling 10 fits
[CV] penalty=l2, max_iter=100, C=0.5 .................................
[CV] penalty=l2, max_iter=100, C=0.5 .................................
[CV] penalty=l1, max_iter=100, C=1.0 .................................
[CV] penalty=l1, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=50, C=0.5 ..................................
[CV] penalty=l2, max_iter=50, C=0.5 ..................................
[CV] penalty=l2, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=100, C=0.1 .................................
[CV] penalty=l2, max_iter=100, C=0.1 .................................
[CV] ..... penalty=l2, max_iter=100, C=1.0, score=0.960, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.5, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.1, score=0.947, total= 0.0s
[CV] ..... penalty=l1, max_iter=100, C=1.0, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=1.0, score=0.973, total= 0.0s
[CV] ...... penalty=l2, max_iter=50, C=0.5, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.1, score=0.893, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.5, score=0.960, total= 0.0s
[CV] ...... penalty=l2, max_iter=50, C=0.5, score=0.960, total= 0.0s
[CV] ..... penalty=l1, max_iter=100, C=1.0, score=0.987, total= 0.0s
[Parallel(n_jobs=12)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=12)]: Done 3 out of 10 | elapsed: 0.0s remaining: 0.1s
[Parallel(n_jobs=12)]: Done 5 out of 10 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=12)]: Done 7 out of 10 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=12)]: Done 10 out of 10 | elapsed: 0.0s finished