Skip to main content
Share Your Experience: Take the 2024 Developer Survey
14 votes
Accepted

How to handle a zero factor in Naive Bayes Classifier calculation?

An approach to overcome this 'zero frequency problem' in a Bayesian setting is to add one to the count for every attribute value-class combination when an attribute value doesn’t occur with every ...
timleathart's user avatar
  • 3,940
13 votes
Accepted

How does Naive Bayes classifier work for continuous variables?

The difference boils down to "how we define $P(x_i|C_k)$?", where $x_i$ is a single feature, and $C_k$ is a class from a total of $K$ classes. Discrete In discrete case, $P(x_i|C_k)$ is ...
Esmailian's user avatar
  • 9,332
7 votes
Accepted

Difference between Bernoulli and Multinomial Naive Bayes

Bernoulli models the presence/absence of a feature. Multinomial models the number of counts of a feature. Here's a concise explanation. Wikipedia warns that Note that a naive Bayes classifier with ...
A. G.'s user avatar
  • 271
7 votes
Accepted

Why does the naive bayes algorithm make the naive assumption that features are independent to each other?

By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint ...
Green Falcon's user avatar
  • 14.1k
6 votes

Does dataset training and test size affect algorithm?

It is completely normal in some circumstances. If you consider the learning problem from a statistical perspective, learning is done by trying to estimate the conditional estimate of your output ...
rapaio's user avatar
  • 4,763
6 votes
Accepted

Can feature importance change a lot between models?

Is this normal? It is not surprising. First, you are using different measures of feature importance. It’s like measuring the importance of people (or simply sorting them) using their a) weight, b) ...
aivanov's user avatar
  • 1,520
5 votes
Accepted

Naive Bayes: Divide by Zero error

Although I haven't verified it, a first glance at the features and training set you used shows an obvious problem. You have just two data samples with the exact same features while you give it ...
Reii Nakano's user avatar
5 votes
Accepted

Is the prediction algorithm absolutely the same for all linear classifiers?

Is it true that linear classifiers differ only in the Learning algorithm, but do they do the same during Prediction y = w1*x1 + w2*x2 + ... + c? Yes, all parametric linear classifiers try to ...
bkshi's user avatar
  • 2,255
5 votes
Accepted

What Shape Does Naive Bayes make?

Specifically talking about Gaussian Naive Bayes, the decision boundary are ellipsoids characterized by the mean and standard deviation of the Gaussian distribution. Image: https://scikit-learn.org/...
Multivac's user avatar
  • 2,989
4 votes

Overfitting Naive Bayes

Let me try to answer your questions point by point. Perhaps you already solved your problem, but your questions are interesting and so perhaps other people can benefit from this discussion. Is Naive ...
Valentin Calomme's user avatar
4 votes

In general, when does TF-IDF reduce accuracy?

The IDF part of TF-IDF gives less weight to a word if it occurs in a large fraction of the documents in your corpus. However, this doesn't necessarily mean that the word is unimportant for ...
Tim Goodman's user avatar
  • 3,100
4 votes

Naive Bayes Should generate prediction given missing features (scikit learn)

Your question is sensible. The way in which posterior probability is calculated in the classical Naive Bayes classifier (in sklearn) is like summation of the conditional probabilities of the all the ...
Arun Aniyan's user avatar
4 votes
Accepted

Handling underflow in a Gaussian Naive Bayes classifier

The standard answer is to work in log space, and manipulate the log of probabilities instead of probabilities, for exactly this reason. This classifier involves products of probabilities which just ...
Sean Owen's user avatar
  • 6,595
4 votes

Does dataset training and test size affect algorithm?

Your training and test errors are affected by the size of the training. Take a look to this plot, usually known as a learning curve: In this example, we compute the training score and the test score (...
Pablo Suau's user avatar
  • 1,787
4 votes
Accepted

Understanding of naive bayes: computing the conditional probabilities

Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$. Then you have $$P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}...
oW_'s user avatar
  • 6,377
4 votes
Accepted

Naive Bayes for SA in Scikit Learn - how does it work

Collecting your data From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of ...
JahKnows's user avatar
  • 8,936
4 votes

Effect of outliers on Naive Bayes

There are different flavors of Naive Bayes, so the answer depends a bit on the use case. One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, ...
oW_'s user avatar
  • 6,377
4 votes

why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class. ...
aranglol's user avatar
  • 2,196
4 votes
Accepted

How to classify a new email as spam/not spam?

I modified your code so the code runs as a block and is setup to predict new data: ...
Brian Spiering's user avatar
4 votes
Accepted

How to improve results from a Naive Bayes algorithm?

Your model certainly overfits. It's likely that the main issue is the inclusion in the features of words which appear very rarely (especially those which appear only once in the corpus): Words which ...
Erwan's user avatar
  • 25.5k
3 votes
Accepted

What Naive Bayes method is being used in this example?

A basic Naive Bayes is being used in this example. Each feature can have a number of different values within the ranges of 2 or 3. Bernoulli Naive Bayes requires that each feature be either true or ...
Joe's user avatar
  • 46
3 votes
Accepted

Bias in Naive Bayes classifier

Unbalanced class distributions First, unbalanced datasets will cause your model to have a bias towards the over-represented classes. If the distribution of the classes is not very drastic then this ...
JahKnows's user avatar
  • 8,936
3 votes
Accepted

Very low probability in naive Bayes classifier

I can't tell you for sure without you describing your calculation more or showing code, but my guess is you're not actually calculating the posterior probability here. I bet this is just the ...
David Marx's user avatar
  • 3,258
3 votes

Doubt in interpretation of Bernoulli Naive Baeyes Algorithm

Bernoulli Naive bayes does not assume gaussian distribution of all continuous features, because it does not make sense. Gaussian Naive Bayes assumes gaussian distribution for continuous features and ...
Christos Karatsalos's user avatar
3 votes
Accepted

Naive Bayes Classifier

We are trying to select the optimal $c$, here $d$ is fixed and hence $P(d)$ and $\frac1{P(d)}$ is just a positive constant. Multiplying an objective function with a positive constant doesn't change ...
Siong Thye Goh's user avatar
3 votes

Naive Bayes Classifier

Because P(d) is constant in terms of c, so it doesn't affect the location of the maximum (only its size but we don't care about that).
Mark.F's user avatar
  • 2,220
3 votes

Avoiding the zero problem

I totally agree with Esmailian. Naive Bayes is Naive - Assumes Independence. Steps: Calculate Independently Smooth using Laplacian smoothing (to avoid zeroing the whole value) Additional Tip: Use ...
William Scott's user avatar
3 votes
Accepted

naive bayes classifier for non-binary feature values

\begin{align} &P(x^{(1)}, \ldots, x^{(m)}, y^{(1)}, \ldots, y^{(m)})\\ &=\prod_{i=1}^m P(x^{(i)},y^{(i)})\\ &= \prod_{i=1}^m P(y^{(i)})P(x^{(i)}|y^{(i)})\\ &= \prod_{i=1}^m \left[\...
Siong Thye Goh's user avatar
3 votes
Accepted

In Naive Bayes classifier how is P(sneezing,builder|flu) = P(sneezing|flu)P(builder|flu)?

This is just a basic property of conditional independence. If two events A and B are conditionally independent, given event C then: $$Pr(A \ and\ B \ |\ C) = Pr(A \ |\ C) * Pr(B \ | \ C)$$ or ...
aranglol's user avatar
  • 2,196
3 votes
Accepted

Genetic algorithms: what connection to support vector machine / naive bayes

A fast search on Google for the term "Genetic Algorithms" will return you this answer: "A genetic algorithm is a search heuristic that is inspired by Charles Darwin's theory of natural evolution" So,...
Pedro Henrique Monforte's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible