I want to build an ensemble model from individual classifiers(e.g KNN,SVM etc) for classification purpose. Before building the ensemble mode, I want to select the best hyperparameter from the individual classifier. To find the best hyperparameter, I used GridSearchCV and the result for different scores is as follows:
- Accuracy: K=1
- F1: K=1
- Precision: K=2
- Recall: K=1
- ROCAUC score: K = 14
There is a difference on the best hyperparameter for different scorings. This happens to other classifiers as well like SVM, DT etc. In this case, how should I decide which value of hyperparameter should I use for building ensemble model and the scores that I should consider most for evaluating the performance of both individual classifier and ensemble model?
Based on the experimental result, I notice that the best hyperparameter value for accuracy & f1 are always similar yet for precision, recall & roc-auc are different from the accuracy & f1.
- features: all categorical data; label : categorical data (values: 1 & 0)
- label 1 = 6157 instances; label 0 = 4898 instances
- The seed of the pseudo random number generator is 42. Used consistently in the cross validation and predictive model.