0
$\begingroup$

I trained a multiclass SVC with RBF kernel on a down-sampled (and therefore balanced) dataset. Now I want to perform grid search to find best cost and gamma.

What performance metric should I optimize for?

I have a highly imbalanced test set. There might be a factor of over 100 between the number of instances of different classes. I am classifying 3D points (car, facade, human) - so I think one could assign equal weight to all classes.

$\endgroup$
0
$\begingroup$

Using resampling methods to fix the imbalanced databases a good method. This can be completed by oversampling, downsampling or creating synthetic instances.

In this kind of problem, precision is not necessarily a good evaluation metric. Instead evaluating True Positives, True Negatives, False Positives and False Negatives is better way to evaluate this.

For this evaluation following techniques can be utilized from Sklearn library.

  • Recall score
  • Accuracy score
  • F1 score
  • AUC Score

In Sklearn the method caled "classification_report" gives a summary of those evaluations (except AUC Score from the list). That can be a good evaluation metric to check the performance of the model.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.