Correct method to report Randomized Search CV results

Question

I have searched online but I still cannot find a definitive answer on how to "correctly" report the results from hyperparameter tuning a machine learning model; though, this may just be some deficiencies with my understanding.

So, I am doing hyperparameter tuning to various machine learning models to try to improve their predictive performance (I'm using RandomizedSearchCV from sklearn in Python to be exact). Would the correct method to report results from the random search be:

• Do the hyperparameter search for, say recall, which then gets the model with the best recall. Then, train this model that has the best recall on the training set, and then test it on the test set to get all of the other metrics (e.g. accuracy, AUC, AUPRC, Brier score, F1, etc)?
• OR do the hyperparameter search for, say recall, but use "multiple metric evaluation" for the metrics I want where the RandomizedSearchCV reports out multiple metrics (e.g. recall, accuracy, AUC, F1, etc.), but searches only for the model with the best recall.

I think that the correct way to do this would be the second method, right? This is because RandomizedSearchCV does cross-validation, so the second method would be the right way, since we aren't doing any cross validation for the "reported" metrics (accuracy, AUC, etc.) except for recall with the first method. I don' know if my understanding about this is correct.

To help, I have put example code for the first and second methods:

First Method:

random_search = RandomizedSearchCV(pipe, param_distributions = param_dist,
                                   n_iter=40, cv=10, scoring = 'recall_macro', random_state=1)
random_search.fit(X_train, y_train)
best_model = random_search.best_estimator_
y_pred = best_model.predict(X_test)
y_train_pred = best_model.predict(X_train)

#Then calculate the metrics from this y_pred and the dataset (y_test, etc.)

Second Method:

#Perform random search
scoring = ['recall_macro', 'accuracy','f1_macro', 'roc_auc']
random_search = RandomizedSearchCV(pipe, param_distributions = param_dist, refit='recall_macro', n_iter=40, cv=10, scoring = scoring, random_state=1)
random_search.fit(X_train, y_train)

#Print metrics
results_dict = random_search.cv_results_
print("Train Accuracy: ", results_dict['mean_train_accuracy'], '+/-', results_dict['std_train_accuracy'])
print("Test Accuracy: ", results_dict['mean_test_accuracy'], '+/-', results_dict['std_test_accuracy'])
print("Test Precision: ", results_dict['mean_test_precision_macro'], '+/-', results_dict['std_test_precision_macro'])
print("Test Recall: ", results_dict['mean_test_recall_macro'], '+/-', results_dict['std_test_recall_macro'])
print("Test F1: ", results_dict['mean_test_f1_macro'], '+/-', results_dict['std_test_f1_macro'])
print("Test AUC: ", results_dict['mean_test_roc_auc'], '+/-', results_dict['std_test_roc_auc'])

Edit: actually, for the second method, I wouldn't even use the test set! So, wouldn't I, for the second method, just fit the random search on the entire dataset (X, y)? There wouldn't be any data leaks, either, right, since everything is in a pipeline and it is doing cross-validation with the random search by default. For this, then I would just change cv=10 to cv=StratifiedKFold(n_splits=10, shuffle=True), so then shuffling is actually performed (one of the requirements for cross validation in general)?