14
votes
Accepted
How to handle a zero factor in Naive Bayes Classifier calculation?
An approach to overcome this 'zero frequency problem' in a Bayesian setting is to add one to the count for every attribute value-class combination when an attribute value doesn’t occur with every ...
- 3,920
13
votes
Accepted
How does Naive Bayes classifier work for continuous variables?
The difference boils down to "how we define $P(x_i|C_k)$?", where $x_i$ is a single feature, and $C_k$ is a class from a total of $K$ classes.
Discrete
In discrete case, $P(x_i|C_k)$ is ...
- 9,247
7
votes
Accepted
Difference between Bernoulli and Multinomial Naive Bayes
Bernoulli models the presence/absence of a feature. Multinomial models the number of counts of a feature. Here's a concise explanation.
Wikipedia warns that
Note that a naive Bayes classifier with ...
- 271
7
votes
Accepted
Why does the naive bayes algorithm make the naive assumption that features are independent to each other?
By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint ...
- 13.9k
6
votes
Does dataset training and test size affect algorithm?
It is completely normal in some circumstances. If you consider the learning problem from a statistical perspective, learning is done by trying to estimate the conditional estimate of your output ...
- 4,643
6
votes
Accepted
Can feature importance change a lot between models?
Is this normal?
It is not surprising.
First, you are using different measures of feature importance. It’s like measuring the importance of people (or simply sorting them) using their a) weight, b) ...
- 1,490
5
votes
Accepted
Naive Bayes: Divide by Zero error
Although I haven't verified it, a first glance at the features and training set you used shows an obvious problem. You have just two data samples with the exact same features while you give it ...
- 291
5
votes
Accepted
Is the prediction algorithm absolutely the same for all linear classifiers?
Is it true that linear classifiers differ only in the Learning
algorithm, but do they do the same during Prediction y = w1*x1 + w2*x2
+ ... + c?
Yes, all parametric linear classifiers try to ...
- 2,215
5
votes
Accepted
What Shape Does Naive Bayes make?
Specifically talking about Gaussian Naive Bayes, the decision boundary are ellipsoids characterized by the mean and standard deviation of the Gaussian distribution.
Image: https://scikit-learn.org/...
- 2,909
4
votes
Overfitting Naive Bayes
Let me try to answer your questions point by point. Perhaps you already solved your problem, but your questions are interesting and so perhaps other people can benefit from this discussion.
Is Naive ...
- 5,456
4
votes
Does dataset training and test size affect algorithm?
Your training and test errors are affected by the size of the training. Take a look to this plot, usually known as a learning curve:
In this example, we compute the training score and the test score (...
- 1,787
4
votes
In general, when does TF-IDF reduce accuracy?
The IDF part of TF-IDF gives less weight to a word if it occurs in a large fraction of the documents in your corpus. However, this doesn't necessarily mean that the word is unimportant for ...
- 3,080
4
votes
Naive Bayes Should generate prediction given missing features (scikit learn)
Your question is sensible. The way in which posterior probability is calculated in the classical Naive Bayes classifier (in sklearn) is like summation of the conditional probabilities of the all the ...
- 343
4
votes
Accepted
SPARK, ML: Naive Bayes classifier often assigns 1 as probability prediction
The reason NB is called "Naive" is that is makes the assumption that the predictive variables are all independent.
This assumption usually skews the model scores (which, under the above naive ...
- 352
4
votes
Accepted
Handling underflow in a Gaussian Naive Bayes classifier
The standard answer is to work in log space, and manipulate the log of probabilities instead of probabilities, for exactly this reason. This classifier involves products of probabilities which just ...
- 6,595
4
votes
Accepted
Understanding of naive bayes: computing the conditional probabilities
Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$.
Then you have $$P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}...
oW_♦
- 6,264
4
votes
Accepted
Naive Bayes for SA in Scikit Learn - how does it work
Collecting your data
From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of ...
- 8,776
4
votes
Effect of outliers on Naive Bayes
There are different flavors of Naive Bayes, so the answer depends a bit on the use case.
One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, ...
oW_♦
- 6,264
4
votes
why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?
I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class.
...
- 1,954
4
votes
Accepted
How to classify a new email as spam/not spam?
I modified your code so the code runs as a block and is setup to predict new data:
...
- 20.5k
4
votes
Accepted
How to improve results from a Naive Bayes algorithm?
Your model certainly overfits. It's likely that the main issue is the inclusion in the features of words which appear very rarely (especially those which appear only once in the corpus):
Words which ...
- 25k
3
votes
Accepted
What Naive Bayes method is being used in this example?
A basic Naive Bayes is being used in this example. Each feature can have a number of different values within the ranges of 2 or 3.
Bernoulli Naive Bayes requires that each feature be either true or ...
- 46
3
votes
Accepted
Bias in Naive Bayes classifier
Unbalanced class distributions
First, unbalanced datasets will cause your model to have a bias towards the over-represented classes. If the distribution of the classes is not very drastic then this ...
- 8,776
3
votes
Accepted
Very low probability in naive Bayes classifier
I can't tell you for sure without you describing your calculation more or showing code, but my guess is you're not actually calculating the posterior probability here. I bet this is just the ...
- 3,208
3
votes
Doubt in interpretation of Bernoulli Naive Baeyes Algorithm
Bernoulli Naive bayes does not assume gaussian distribution of all continuous features, because it does not make sense. Gaussian Naive Bayes assumes gaussian distribution for continuous features and ...
3
votes
Accepted
Naive Bayes Classifier
We are trying to select the optimal $c$, here $d$ is fixed and hence $P(d)$ and $\frac1{P(d)}$ is just a positive constant.
Multiplying an objective function with a positive constant doesn't change ...
- 3,003
3
votes
Naive Bayes Classifier
Because P(d) is constant in terms of c, so it doesn't affect the location of the maximum (only its size but we don't care about that).
- 2,200
3
votes
Avoiding the zero problem
I totally agree with Esmailian.
Naive Bayes is Naive - Assumes Independence.
Steps:
Calculate Independently
Smooth using Laplacian smoothing (to avoid zeroing the whole value)
Additional Tip:
Use ...
- 299
3
votes
Accepted
naive bayes classifier for non-binary feature values
\begin{align}
&P(x^{(1)}, \ldots, x^{(m)}, y^{(1)}, \ldots, y^{(m)})\\
&=\prod_{i=1}^m P(x^{(i)},y^{(i)})\\
&= \prod_{i=1}^m P(y^{(i)})P(x^{(i)}|y^{(i)})\\
&= \prod_{i=1}^m \left[\...
- 3,003
Only top scored, non community-wiki answers of a minimum length are eligible