Excuse me if this is a simple question (I'm a newbie to data science). I am trying to implement a Naive Bayes classifier in Python.

I have implemented a standard Naive Bayes classifier (Bernoulli) and this went very well. Suppose I have 50,000 samples where the first half is labelled '0' and the second half '1'. Then the shape of the X_train matrix will be (#number_of_unique_words/features, 50000) and shape of y_train (50000,).

From how I understand Naive Bayes, I can now also join the first 25000 documents to represent one big document with label '0' and one big document with label '1'. However, this gives me very different results when I try to predict the test set. Is my understanding of Naive Bayes correct or is my implementation just wrong?

  • $\begingroup$ Are you training on word occurrence (either 0 or 1) or on word frequency (any integer)? $\endgroup$ – Elias Strehle Feb 19 '18 at 19:18

Your assumption seems to be correct that both approaches are actually equivalent. I can not see any difference here in the propabilities and conditional propabilities for either one way or the other.

That's why the predictions should also be the same for both ways. It is more likely that there is something wrong with your implementation.

| improve this answer | |
  • $\begingroup$ Indeed the error was in the implementation! $\endgroup$ – henk.henkert Aug 8 '18 at 10:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.