I am working on a regression problem where I have a lot of outliers in multiple variables. As far as I can think of, there are 3 things I can do to outliers.

  1. Remove them (least attractive option)

  2. Transform them (log transformation, box-cox transformation etc)

  3. Do nothing and build a model including them

My question is regarding the second point. If I want to transform my features using any of the transformations solely for the purpose of outlier, is it ok to do it?

  • $\begingroup$ There is a 4th option, use ML model that is less sensitive to outliers, for example Random Forest is less sensitive to outliers than OLS. $\endgroup$
    – Akavall
    Nov 18 '21 at 19:43
  • $\begingroup$ Yes I can and will do that but I just had this question that if a model is sensitive to outliers, can I use transformations like mentioned above solely for outlier treatment? $\endgroup$
    – spectre
    Nov 19 '21 at 5:01

Although it is the least attractive, the best solution is to eliminate them. Including outliers, even if modified, goes a long way in modifying your dataset. For example, if your goal is to build a Machine Learning model, using modified data falsifies the training of your model and therefore gives you an unreliable result.

The whole thing is summarized by the principle "garbage in, garbage out", or if you use garbage data as input you will get garbage results. Therefore the cleanliness of the data is very important, better less data than more but not very reliable data.

  • $\begingroup$ I don't agree with the statement "transforming the data makes it garbage data". Outliers just represent the extreme part of a dataset and are not garbage/invalid values. Also If I drop each and every outlier from all my features, there'd be a huge amount of info loss (not tot mention I'm already working with a small dataset). $\endgroup$
    – spectre
    Nov 18 '21 at 14:11
  • $\begingroup$ If there are so many outliers, are you really sure that they are outliers? Couldn't it simply be another cluster? $\endgroup$
    – Inuraghe
    Nov 18 '21 at 14:24
  • $\begingroup$ No I checked (using more than one test) and they are outliers. $\endgroup$
    – spectre
    Nov 18 '21 at 15:56
  • $\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Nov 18 '21 at 23:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.