I want to build an ensemble model from individual classifiers(e.g KNN,SVM etc) for classification purpose. Before building the ensemble mode, I want to select the best hyperparameter from the individual classifier. To find the best hyperparameter, I used GridSearchCV and the result for different scores is as follows:

e.g KNN

  • Accuracy: K=1
  • F1: K=1
  • Precision: K=2
  • Recall: K=1
  • ROCAUC score: K = 14

There is a difference on the best hyperparameter for different scorings. This happens to other classifiers as well like SVM, DT etc. In this case, how should I decide which value of hyperparameter should I use for building ensemble model and the scores that I should consider most for evaluating the performance of both individual classifier and ensemble model?

Based on the experimental result, I notice that the best hyperparameter value for accuracy & f1 are always similar yet for precision, recall & roc-auc are different from the accuracy & f1.

Dataset info:

  • features: all categorical data; label : categorical data (values: 1 & 0)
  • label 1 = 6157 instances; label 0 = 4898 instances
  • The seed of the pseudo random number generator is 42. Used consistently in the cross validation and predictive model.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.