1
$\begingroup$

I am using MLPClassifer example from scikit-learn

The code for training:

from sklearn.neural_network import MLPClassifier
X = [[0., 0.], [1., 1.]]
y = [0, 1]
clf = MLPClassifier(solver='lbfgs', alpha=1e-5,
                    hidden_layer_sizes=(5, 2), random_state=1)

clf.fit(X, y)                         

At the predict step, we use test data [2., 2.], [-1., -2.] in clf.predict([[2., 2.], [-1., -2.]]). The output of this function is array([1, 0])

As we observe, the test data [2.,2.] is not in the train dataset we passed. Still, we got the closest match as label 1.

What i am trying to find is if the test data i supplied is not in the train dataset, i should print a message to user that data is not valid instead of telling him the wrong label as 1.

For instance, in knn classification, i have kneighbours function which tells the distance of my closest neighbours to the test data i supplied in a 0 to 1 scale. So, i could easily eliminate the test data samples which are highly distant from my train data samples by keeping threshold at 0.6 or 0.7.

Is there any criteria/threshold like this i could do with MLPClassifier or with any one of Incremental Classifiers mentioned here which can restrict my test samples if not present in train dataset ?

Question migrated from SO

$\endgroup$
1
$\begingroup$

SGDClassifier has desicion_function which tells the distance to the hyperplane, where the values are compared to.

This value could imply too big and too low values.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks mico. I see that decision function have values in negative scale, 0 and positive scale. when i do decive_function(untrained_sample), it is showing all negative values in comparision with all the trian data. So, can i be sure that the unknown_sample i passed is always bearing a negatve value and set threshold to consider only those values above 0 for a close match ? $\endgroup$ – user1 Mar 7 '18 at 16:58
  • $\begingroup$ I am not sure. You could test with four values: one above the 1, one below 0 and two between them, one closer to 1 and one closer to 0. And of course for comparison 1 and 0. Then you have the cases tested for sure. Put these as inputs and make observations. $\endgroup$ – mico Mar 7 '18 at 17:24
  • $\begingroup$ Thanks mico. For fit, all closest samples are above 0. So, i can keep threshold above 0. But for partial_fit(), it is not the case. The scale is varying. The untrained samples also hold positive values (above 0). Have to play around to find the exact number incase of partial_fit(). Any ideas for partial_fit() ? $\endgroup$ – user1 Mar 9 '18 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.