3
$\begingroup$

I'm trying to make a partial fitting with GuassianNB here's small snippet of my code

classfier = GaussianNB()
classfier.fit(X_init, y_init)
for i in range(X_train.shape[0]):
    if('some condition'):
        classfier = GaussianNB() 
        classfier.partial_fit(X_train[i, :].reshape(1, -1), y_train[i].reshape(1, -1).ravel(), classes = np.unique(y_train))
    else:
        classfier.partial_fit(X_train[i, :].reshape(1, -1), y_train[i].reshape(1, -1).ravel())

Everything works correctly until the condition is true, it seems that the classifier for some reason stops learning and starts making arbitrary predictions (predicts all zeros)

tn, fp, fn, tp = confusion_matrix(y_test, pred).ravel()
print(tn, fp, fn, tp)

>>> 1324 0 1031 0

I thought that maybe my classifier made predictions after reinitializing it (the condition is satisfied right before the loop ends) but after I checked it, it made at least 900 partial fittings before the loop ends with a both 0 and 1 labels.

I'm so confused, what's happening? Thanks

Edit: I figured out what's wrong

it turned out that the problem was that my dataset has so many features (around 40000 as it was originally a string of reviews and vectorized it with tfidf) And for some reason that made GaussianNB classifier break down. I'm trying to know why GaussianNB doesn't with a big number of features. If anyone knows, I would appreciate the help. Thanks

$\endgroup$
0
$\begingroup$

It is not the number of features that is the problem with Gaussian Naive Bayes (GaussianNB). It is the decision boundary that GaussianNB is learning. Naive Bayes is constrained to the learn the marginal distribution of the data because "naive" assumption. Often times the conditional distribution is useful to make predictions. Given the performance of GaussianNB on that dataset, it might make sense to apply a different classifier that can learn a different decision boundary.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.