# Two questions on hyper-parameter tuning

Question 1:

In the example of logistic regression, I often see the regularization constant and penalty methods being tuned by a grid search. However, it seems like there are a lot more options for tuning:

classifier_os.get_params()
gives:
{'C': 1.0,
'class_weight': None,
'dual': False,
'fit_intercept': True,
...
and many more!


So my question is: Are these other parameters typically not worth tuning, or are they left out in examples for another reason? For example, I changed to solver='liblinear' and got sub-par results compared to lbfgs. Is that a coincidence, or are the other parameters already at their best by default?

Question 2:

Does performing grid search on hyper-parameters guarantee improved performance when tested on the same data set?

I ask because my intuition was "yes", however I got slightly lower scores after tuning my regularization constant:

p_grid = {
'C': np.logspace(-3, 3, 7)
}
clf = model_selection.GridSearchCV(classifier_os, p_grid, scoring='accuracy')
clf.fit(x_train, y_train)
y_pred = clf.predict(x_test)
metrics.classification_report(y_pred, y_test, output_dict=True)


Gives me the following scores (C=10):

accuracy :  0.8218181818181818
macro avg:
precision :  0.8210875331564986
recall :  0.8213603058298822
f1-score :  0.8212129655428624
support :  275


As compared to before tuning (C=1):

accuracy :  0.8290909090909091
macro avg:
precision :  0.8287798408488063
recall :  0.8285358354537744
f1-score :  0.8286468069310212


The only thing that the tuning changed was to make the regularization constant C=10 instead of the default C=1