0
$\begingroup$

Consider a neural with N layers of size $M_n$. I want this NN to do nothing but memorize. I want it to fail if it is asked to make a classification for an input it has never seen before, I want it to fail in some way that I can detect. Is this possible?

Edit: someone has said to use a dictionary. That is a good suggestion except for something I did not put in the original question. I want to leave open the possibility that, even though it is memorizing, I want the neural network to also discover latent features, to have hidden layers capable of discovering latent features. This sounds impossible, but I put it out there.

$\endgroup$

3 Answers 3

2
$\begingroup$

As I understand correctly, you have a finite number of permittable inputs and you don't want to perform a prediction for any input outside this set. In that case, I reckon that you don't need a neural network and probably you should use a dictionary. We use neural networks as a model that generalizes and its value flows from using them on, even slightly, unknown data.

If you really want to use a neural network you can use a dictionary with inputs to raise an error when the input is not in the dictionary before you perform the prediction. On the other hand, if you want to avoid extrapolation, you might need an additional model to predict whether your data are in the known scope.

$\endgroup$
0
0
$\begingroup$

fail ... for an input it has never seen before

I want the neural network to also discover latent features

You're asking the NN to do something it's not good at. There's an easy way out -- just give it the answer! Use a dictionary to determine if we're looking at a novel input or not. And feed that indicator variable to the NN as a synthetic feature. The net will soon learn to produce a "fail!" $Y_1$ output on novel inputs. Additionally it's learning about the structure of all inputs, which it can use for whatever $Y_2$ and $Y_3$ outputs you're training it to produce.

$\endgroup$
0
$\begingroup$
if feature values seen before
    Use neural network prediction 
else 
    Fail

This seems to accomplish all facets of what you want. You can put it in a function.

def predictor(feature values)
    If feature values seen before
        return neural net prediction
    Else
        Fail

This isn’t real Python code but should give the idea of what to do.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.