0
$\begingroup$

When training an algorithm, I have ROC = 0.5896, Sensitivity = 0.3333, and Specificity = 0.8375. When considering the test set, Sensitivity = 1 and Specificity = 1. This could happen or is a problem?

Another question: When comparing several learning algorithms, I choose the best based on the results of the test set?

Any help will be appreciated.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
2
$\begingroup$

Seems quite strange. I bet you've got a bug or test set size is small enough. Like if there're only 2 objects: 1 positive and 1 negative, you could have classified them correctly even with a random classifier. I assume your test set is not that small, which means there's a bug somewhere. Also, in theory, if both of your sets are large enough and are drawn from the same data distribution, you can usually have one of 2 types of behavior:

  1. about the same result both on training and testing sets (when your model underfits or fits well)
  2. better results on the training set then on the testing (when your model if overfit)
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.