1
$\begingroup$

I work for a highly-regulated entity, so I have to obfuscate what I'm working on; I'll provide the following as examples on what I'm doing.

I am training an XGB model for NLP comments about breeds of dogs (Tensorflow, sklearn). I have a dataset about dogs that I'm using to train the algo and cross-validation testing is promising.

However, my test data is not just about dogs, it's about animals in general. So that data can be about dogs, cats, elephants, whatever. Right now, the XGB model is giving all of the animals it does not recognize a default value (a category). How can I make my XGB just say "I don't know" instead? Is there a way to assign a default value to test values that don't make sense in any category?

BTW, I realize that another possibility is to create a model to classify the test data but I want to see if there is a way to solve this within the XGB model.

$\endgroup$
  • $\begingroup$ What category you have defined for animals other than dogs? $\endgroup$ – Ankit Seth Feb 10 '18 at 10:31
1
$\begingroup$

You could e.g. use as objective "multi:softprob" instead of "multi:softmax". The model is identical, but you get a probability for each label instead if the most probable label. Then you can specify a threshold, e.g. 0.5 and return "unknown" if no probability is above the limit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.