I have an unbalanced dataset which has 920 samples in total, 689 belong to the first class, and 222 to second class. and both classes are significant for me. so when building a classifier model such as SVM or KNN. what measurement should I consider to evaluate the performance of the classifier? usually people use accuracy. but in my case some times I get high accuracy but zero specificity which clearly indicates that the class is biased towards the majority class (class one in my case). I've been advised to use the F-score which combines both specificity and sensitivity. Also, there is the AUC. so what do you suggest?
Useful metrics in such scenario are:
Few articles on how to choose metrics for a specific project are:
Evaluation Metrics, ROC-Curves and imbalanced datasets by David S. Batista,
What metrics should be used for evaluating a model on an imbalanced data set? by Shir Meir Lador,