3
$\begingroup$

Let's say I train a Neural Net, and I have a Categorical Feature X.

During training, there are only 3 classes seen in feature X; A, B, C.

Now, let's say I want to make predictions from this trained model - but there is now a fourth class, D, that appears in feature X.

How does the Neural Net handle this? It hasn't been trained on any examples belonging to class D?

$\endgroup$
2
  • 1
    $\begingroup$ "Open set" is the keyword you want. Google "open set learning/recognition/classification". $\endgroup$ Apr 15 at 23:38
  • $\begingroup$ There is no "the Neural Net". You can choose among many many different architectures. $\endgroup$ Apr 16 at 18:42

1 Answer 1

7
$\begingroup$

Typically feature X would be presented to the NN via one hot encoding. So we have three (boolean) indicator variables, denoting the presence of A, B, or C.

If the problem domain admits of D or E values there is simply no way to represent them, without redoing the whole representation. At which point we would likely retrain.

If we anticipate that novel classes could arise in future, we might choose to model a "miscellaneous" class from the outset. Then D and E could be assigned the misc. label when we eventually encounter them.


Here is a separate difficulty. Imagine that X is our most informative feature. And we never observe an instance of class B in the training data.

Then at inference time the model is unlikely to have good predictive power when it encounters out-of-sample instances which do contain B.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.