1
$\begingroup$

I work for a highly-regulated entity, so I have to obfuscate what I'm working on; I'll provide the following as examples on what I'm doing.

I am training an XGB model for NLP comments about breeds of dogs (Tensorflow, sklearn). I have a dataset about dogs that I'm using to train the algo and cross-validation testing is promising.

However, my test data is not just about dogs, it's about animals in general. So that data can be about dogs, cats, elephants, whatever. Right now, the XGB model is giving all of the animals it does not recognize a default value (a category). How can I make my XGB just say "I don't know" instead? Is there a way to assign a default value to test values that don't make sense in any category?

BTW, I realize that another possibility is to create a model to classify the test data but I want to see if there is a way to solve this within the XGB model.

Improve this question
$\endgroup$
1
  • $\begingroup$ What category you have defined for animals other than dogs? $\endgroup$
    – Ankit Seth
    Feb 10, 2018 at 10:31

1 Answer 1

Reset to default
1
$\begingroup$

You could e.g. use as objective "multi:softprob" instead of "multi:softmax". The model is identical, but you get a probability for each label instead if the most probable label. Then you can specify a threshold, e.g. 0.5 and return "unknown" if no probability is above the limit.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.