0
$\begingroup$

Can I leave unedited natural outliers in a dataset (outliers that have not appeared just because of mistyping of mistakes in the data)? Or should I also remove them or change them?

$\endgroup$

1 Answer 1

0
$\begingroup$

Yes you should keep the natural outliers in a dataset. They represent an extreme end of the data you have and contain useful info. They also help you with anomaly detection if you wish.

But it also depends on the type of problem at hand. If for example in the case of Titanic dataset, where we are classifying who survived and who didn't. It is ok to remove the outliers as removing them won't be detrimental to the result. The passengers are already dead and removing the outliers won't lead to some serious loss.

On the other hand in the case of classifying weather a patient has a tumor or not, removing the outliers would be a bad idea, as it will lead to misclassifying and ultimately incorrect diagnosis/treatment.

If you are certain that these outliers are because of mistyping then you can safely remove them, but only if you are certain that they are because of mistyping. Else for a real world problem, it is always wise to keep the outliers.

Cheers!

$\endgroup$
2
  • $\begingroup$ Your argument about the Titanic data set does not make sense to me. It seems like you are okay with removing outliers there because you think the Titanic data are not important, whereas a medical diagnosis is important. $\endgroup$
    – Dave
    Commented Dec 24, 2021 at 16:51
  • $\begingroup$ @Dave The outliers in the Titanic data do not have an impact on that particular use case. Hence we can remove them if we want to. Whereas in the case of cancer or tumor detection, outliers are extremely important data points because not everyone has cancer. The number of people having cancer is extremely low and so those data points can also be an outlier because those values are completely different. $\endgroup$
    – spectre
    Commented Dec 24, 2021 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.