3
$\begingroup$

I have in mind a situation where a Decision Tree is trained with a dataset where one category has got just three possible values: A, B and C.

So as I understand the node for this category will have three splits: A, B and C.

What will happen in case an observation with a null for this feature arrives, or if in the testing set appears a value D for this category?

Does Decision Tree leave one of the splits as default to handle this situations?

$\endgroup$
7
  • $\begingroup$ Welcome to the site! Obviously your model cannot. To understand it better lets take another example: you built a classifier to classify left and right hand but if you give leg photo what would the model do? It cannot classify it might classify but cannot expect it to be correct. As long as the model isn't trained on something which is out of its scope it cannot classify like D. It will throw an error like model is not trained on D. $\endgroup$ – Toros91 Jul 4 '18 at 7:13
  • $\begingroup$ If there are values in testing set that are not appearing in training set then your sampling of data is not correct hence your model which is trained on the training data is not complete. Because there is a gap between the sample and population of the data. $\endgroup$ – Kaustubh Jul 4 '18 at 7:51
  • $\begingroup$ @Toros91 Ok, Thank you guys. It is what I was fearing, that for an unkown category it will fail. However I though about missing data, which may appear once deployed and already predicting on run-time, it may select a default branch to handle that situation. What I take from your explanations is that I should handle missing data once in production before trying to predict a value. Thanks for your time. $\endgroup$ – Ignacio Alorre Jul 4 '18 at 8:04
  • 1
    $\begingroup$ @Kaustubh Thanks. This sort of question should be removed from the site? If it is very basic it may not be helpful for other users so it may be no point in keeping it here. $\endgroup$ – Ignacio Alorre Jul 4 '18 at 8:05
  • 2
    $\begingroup$ This is a very legitimate question and it's not as clear cut as you guys make it out to be now, I will write a more thorough answer. $\endgroup$ – Jan van der Vegt Jul 4 '18 at 8:21
6
$\begingroup$

There are two perspectives to this question, from a mathematical or machine learning perspective and from a technical perspective. From a technical perspective this depends on the implementation of the decision tree. An unseen value is not that different from a missing value, and sklearn for example does not deal with unknown values well and will fail on unseen or unknown values. Other tree implementations might deal better with this.

From a mathematical or ML perspective, without making some assumptions you cannot solve this problem of course. However an assumption you could make is that unknown or unseen values are average values. With that assumption you could follow both paths on the split where the unknown value occurs. Then you collect both results from the split and weigh them by the number of training samples that happened there. That way you can still make a prediction for these unknown values.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.