# Tag Info

Accepted

### How to handle a zero factor in Naive Bayes Classifier calculation?

An approach to overcome this 'zero frequency problem' in a Bayesian setting is to add one to the count for every attribute value-class combination when an attribute value doesn’t occur with every ...
Accepted

### How does Naive Bayes classifier work for continuous variables?

The difference boils down to "how we define $P(x_i|C_k)$?", where $x_i$ is a single feature, and $C_k$ is a class from a total of $K$ classes. Discrete In discrete case, $P(x_i|C_k)$ is ...
Accepted

### Difference between Bernoulli and Multinomial Naive Bayes

Bernoulli models the presence/absence of a feature. Multinomial models the number of counts of a feature. Here's a concise explanation. Wikipedia warns that Note that a naive Bayes classifier with ...
Accepted

### Why does the naive bayes algorithm make the naive assumption that features are independent to each other?

By doing so, the joint distribution can be found easily by just multiplying the probability of each feature whilst in the real world they may not be independent and you have to find the correct joint ...

### Does dataset training and test size affect algorithm?

It is completely normal in some circumstances. If you consider the learning problem from a statistical perspective, learning is done by trying to estimate the conditional estimate of your output ...
Accepted

### Can feature importance change a lot between models?

Is this normal? It is not surprising. First, you are using different measures of feature importance. It’s like measuring the importance of people (or simply sorting them) using their a) weight, b) ...
Accepted

### Naive Bayes: Divide by Zero error

Although I haven't verified it, a first glance at the features and training set you used shows an obvious problem. You have just two data samples with the exact same features while you give it ...
Accepted

### Is the prediction algorithm absolutely the same for all linear classifiers?

Is it true that linear classifiers differ only in the Learning algorithm, but do they do the same during Prediction y = w1*x1 + w2*x2 + ... + c? Yes, all parametric linear classifiers try to ...
Accepted

### What Shape Does Naive Bayes make?

Specifically talking about Gaussian Naive Bayes, the decision boundary are ellipsoids characterized by the mean and standard deviation of the Gaussian distribution. Image: https://scikit-learn.org/...

### Overfitting Naive Bayes

Let me try to answer your questions point by point. Perhaps you already solved your problem, but your questions are interesting and so perhaps other people can benefit from this discussion. Is Naive ...

### Does dataset training and test size affect algorithm?

Your training and test errors are affected by the size of the training. Take a look to this plot, usually known as a learning curve: In this example, we compute the training score and the test score (...

### In general, when does TF-IDF reduce accuracy?

The IDF part of TF-IDF gives less weight to a word if it occurs in a large fraction of the documents in your corpus. However, this doesn't necessarily mean that the word is unimportant for ...

### Naive Bayes Should generate prediction given missing features (scikit learn)

Your question is sensible. The way in which posterior probability is calculated in the classical Naive Bayes classifier (in sklearn) is like summation of the conditional probabilities of the all the ...
Accepted

### SPARK, ML: Naive Bayes classifier often assigns 1 as probability prediction

The reason NB is called "Naive" is that is makes the assumption that the predictive variables are all independent. This assumption usually skews the model scores (which, under the above naive ...
Accepted

### Handling underflow in a Gaussian Naive Bayes classifier

The standard answer is to work in log space, and manipulate the log of probabilities instead of probabilities, for exactly this reason. This classifier involves products of probabilities which just ...
Accepted

### Understanding of naive bayes: computing the conditional probabilities

Your formula is correct for one $w_i$, but if you want to classify a document, you need to compute $P(c | w_1,\ldots,w_N)$. Then you have P(c | w_1,\ldots,w_N) = \frac{P(c)\cdot P(w_1,\ldots,w_N|c)}...
Accepted

### Naive Bayes for SA in Scikit Learn - how does it work

Collecting your data From the comments you state that you wish to classify comments into a label (1-poor, 2-fair, 3-ok, 4-good, 5-very good). Thus you will be training a model that maps a set of ...

### Effect of outliers on Naive Bayes

There are different flavors of Naive Bayes, so the answer depends a bit on the use case. One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, ...

### why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class. ...
Accepted

### How to classify a new email as spam/not spam?

I modified your code so the code runs as a block and is setup to predict new data: ...
Accepted

### How to improve results from a Naive Bayes algorithm?

Your model certainly overfits. It's likely that the main issue is the inclusion in the features of words which appear very rarely (especially those which appear only once in the corpus): Words which ...
Accepted

### What Naive Bayes method is being used in this example?

A basic Naive Bayes is being used in this example. Each feature can have a number of different values within the ranges of 2 or 3. Bernoulli Naive Bayes requires that each feature be either true or ...
Accepted

### Bias in Naive Bayes classifier

Unbalanced class distributions First, unbalanced datasets will cause your model to have a bias towards the over-represented classes. If the distribution of the classes is not very drastic then this ...
Accepted

### Very low probability in naive Bayes classifier

I can't tell you for sure without you describing your calculation more or showing code, but my guess is you're not actually calculating the posterior probability here. I bet this is just the ...

### Doubt in interpretation of Bernoulli Naive Baeyes Algorithm

Bernoulli Naive bayes does not assume gaussian distribution of all continuous features, because it does not make sense. Gaussian Naive Bayes assumes gaussian distribution for continuous features and ...
Accepted

### Naive Bayes Classifier

We are trying to select the optimal $c$, here $d$ is fixed and hence $P(d)$ and $\frac1{P(d)}$ is just a positive constant. Multiplying an objective function with a positive constant doesn't change ...

### Naive Bayes Classifier

Because P(d) is constant in terms of c, so it doesn't affect the location of the maximum (only its size but we don't care about that).