I know that outliers are present in data but their behaviour varies a lot from remaining data points. But today while learning about naive-Bayes they mentioned that naive-Bayes can affected by the outliers. But which points in data set are termed as outliers and how do we identify them?
I define an outlier in the following ways.
- It can be a wrong data entry (Eg. human typing error)
- It can be a data that has values that are not relevant (Eg. an entry of total which is calculated as the sum of the above columns. This data can be misleading at times so it should be removed)
- It can be a data entry that is all or most fields blank (Eg. a row in the data where all fields are blank. This row maybe not contributing anything to the analysis)
- It may be extreme values which fall way out of the range of the other data (Eg. when we are calculating the age of Humans, anyone with age (say) above 120 years is an extreme case and that can be ignored depending upon our analysis goal)
In my view, the words which are not seen in training data can be considered as outliers as it leads to the probability of the word to zero, in case of naive bayes.
Also,I think too frequent and too rare words in the corpus can also be considered as outliers as they effect the model.