2
$\begingroup$

We say Bernoulli naive bayes assumes gaussian distribution of all continuous features. What happens if I have categorical features also in the dataset?

What type of prior transformation in data is suitable before applying Bernoulli naive baeyes to a dataset ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

Bernoulli Naive bayes does not assume gaussian distribution of all continuous features, because it does not make sense. Gaussian Naive Bayes assumes gaussian distribution for continuous features and it is the appropriate way for using Naive Bayes approach if you have continuous features.

On the other hand, if you have binary categorical data then the appropriate approach is Bernoulli Naive Bayes. If your features are categorical but not binary then you could transform them into binary categorical using dummy boolean variables for each available value of the categorical features. The main point of Naive Bayes algorithm is the assumption of feature independence, which in some real world classification problems does not hold.

You need to specify a conditional probability p(x|y) of the feature value x given the class label y. Since Naive Bayes assumes that all features are conditionally independent given the class, you can mix different likelihood models for each feature considering any prior knowledge about it.

For example, considering a continuous feature you might assume that p(x|y) is normally distributed, then you can stimate the mean and variance for this feature under each class in the training set and after that you can use the PDF of the Normal Distribution to estimate p(x|y). Considering another feature which is categorical, you can estimate p(x|y) using a Bernoulli or multinomial event model and multiply the two conditional probabilities together in the final prediction (since they are assumed to be independent anyway).

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ What happens if I have both continuous and categorical in a dataset. Which form of Naive Baeyes should be used then? $\endgroup$
    – Sahil Chaturvedi
    May 26, 2018 at 4:19
  • $\begingroup$ Check please the updated answer above. $\endgroup$
    – Christos Karatsalos
    May 26, 2018 at 10:23
  • $\begingroup$ @ChristosKaratsalos You mention that categorical variables can be transformed into dummy boolean variables. Is this required? Also won't this break the independence assumption between features (only one of the dummy variables will be 1, hence they are not independent)? I guess this is mentioned in the context of Bernoulli Naive Bayes. In the second part of your answer, you mention mixing different likelihood models, so this is a more generic approach. $\endgroup$
    – raghu
    May 26, 2018 at 13:23
  • $\begingroup$ I mentioned the tranformation of the categorical variables considering that for some reason you want to apply the Bernoulli Naive Bayes. Otherwise you could apply the generic approach that i ve described, which is the appropriate when you have both numeric and categorical data. The assumption of feature independence does not hold in many cases in real world problems, so it is one of the disadvantages of Naive Bayes Algorithm. $\endgroup$
    – Christos Karatsalos
    May 26, 2018 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.