Understand RandomizedSearchCV output

Question

I am trying to do hyperparameter tunning for the Randomforest regression model. I'm using RandomizedSearchCV (scikit-learn) and I defined verbose=10.
For that reason, I'm getting messaged while it's running and I would like to understand them a bit better.

Those are my parameters for RandomizedSearchCV:

from joblib import Parallel, delayed, parallel_backend
# Use the random grid to search for best hyperparameters
# First create the base model to tune
rf = RandomForestRegressor(-1)


with parallel_backend('threading',n_jobs=12):
    rf_random = RandomizedSearchCV(estimator = rf, param_distributions = random_grid, n_iter = 12, cv = 3, verbose=10, random_state=42)#, n_jobs = -2)
    # Fit the random search model
    rf_random.fit(x_train, y_train)

After running a while )and is still running) I have those messages:

my questions are:
1. what are the rows before the first red row? I thought it may be the combinations of parameters but that doesn't make a lot of sense because those are not enough
2. What is the meaning of the row between the red rows? Why those parameters are there? is this after one CV?
3. "Done one task" - what is the task? one CV?
4. Why do I get score only to some of the parameters?

Answer 1

1. what are the rows before the first red row? I thought it may be the combinations of parameters but that doesn't make a lot of sense because those are not enough

Parameters, which are the candidates of the CV are printed

2. What is the meaning of the row between the red rows? Why those parameters are there? is this after one CV?

It is printed after the completion of the 3rd(as defined in your code) fold for a parameter Candidate

3. "Done one task" - what is the task? one CV?

This is from the joblib I believe. It's the completion of one fold i.e. one out of 36.

4. Why do I get a score only for some of the parameters?

The score is printed after the completion of 3(as defined in your code) fold of each parameter candidate

In a multi-thread setup, we can't rely on the sequence of the messages as it will depend on the Thread allocation.

My answer is based on this small setup.

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import RandomizedSearchCV
iris = load_iris()
logistic = LogisticRegression(solver='saga', tol=1e-2, random_state=0)

distributions = dict(C=[0.1,0.5,1.0],
                      penalty=['l2', 'l1'], max_iter=[50,100])
clf = RandomizedSearchCV(logistic, distributions, random_state=0,n_iter = 5, cv = 2, verbose=10)

from joblib import Parallel, delayed, parallel_backend
with parallel_backend('threading',n_jobs=12):
    search = clf.fit(iris.data, iris.target)

Output

Fitting 2 folds for each of 5 candidates, totalling 10 fits
[CV] penalty=l2, max_iter=100, C=0.5 .................................
[CV] penalty=l2, max_iter=100, C=0.5 .................................
[CV] penalty=l1, max_iter=100, C=1.0 .................................
[CV] penalty=l1, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=50, C=0.5 ..................................
[CV] penalty=l2, max_iter=50, C=0.5 ..................................
[CV] penalty=l2, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=100, C=1.0 .................................
[CV] penalty=l2, max_iter=100, C=0.1 .................................
[CV] penalty=l2, max_iter=100, C=0.1 .................................
[CV] ..... penalty=l2, max_iter=100, C=1.0, score=0.960, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.5, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.1, score=0.947, total= 0.0s
[CV] ..... penalty=l1, max_iter=100, C=1.0, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=1.0, score=0.973, total= 0.0s
[CV] ...... penalty=l2, max_iter=50, C=0.5, score=0.973, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.1, score=0.893, total= 0.0s
[CV] ..... penalty=l2, max_iter=100, C=0.5, score=0.960, total= 0.0s
[CV] ...... penalty=l2, max_iter=50, C=0.5, score=0.960, total= 0.0s
[CV] ..... penalty=l1, max_iter=100, C=1.0, score=0.987, total= 0.0s

[Parallel(n_jobs=12)]: Using backend ThreadingBackend with 12 concurrent workers.
[Parallel(n_jobs=12)]: Done 3 out of 10 | elapsed: 0.0s remaining: 0.1s
[Parallel(n_jobs=12)]: Done 5 out of 10 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=12)]: Done 7 out of 10 | elapsed: 0.0s remaining: 0.0s
[Parallel(n_jobs=12)]: Done 10 out of 10 | elapsed: 0.0s finished