Suppose you have some training dataset that you want to use to train some ML models, where targets are comprised between let's say 1 and 100. However, from the 4000 samples, there are few of them (less than 10) which have values out of the previous range, much higher than 100, let's say 300. Is it reasonable to ignore these samples and remove them from the data set, or should they be kept ? I saw people react differently, some of them say that they may hurt the model, while others say no as these samples give additional information to the model.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It mostly depends on what you are trying to achieve with your model. Sometimes the information carried by outliers is indeed negligible if not of interest (say, for example, that those high values are caused by data collection/input errors), and may affect your model performance. In some other cases though, outliers carry a lot of meaning and you might want your model to be aware of their existence/possibility. In other scenarios, the outliers are what you actually care about (see Anomaly/Novelty detection e.g.).
Long story short, if these outliers are really such (i.e. they appear with a very low frequency and very likely are bad/random/corrupted measurements) and they do not correspond to potential events/failures that your model should be aware of, you can safely remove them. In all other cases you should evaluate case by case what those outliers represent.
