I want to predict real estate prices using several Machine Learning algorithms. My dataset contains numerical and categorical predictors. I already eliminated the outliers of numerical variables. Now I'm wondering on how to deal with "outliers" (i.e., imbalanced classes) of the categorical variables but I could not find anything on this topic. Do I have to deal with the imbalanced classes (outliers) at all or is it only relevant for classification tasks?

Side note, if important: I encoded the categorical variables using one-hot encoding.

  • $\begingroup$ You're not supposed to eliminate "outliers"! Regardless if it's regression or classification, or if the variables are numerical or categorical. $\endgroup$ Commented Jul 15, 2022 at 10:24
  • $\begingroup$ @user2974951 could you explain this please? Why should I not eliminate outliers? $\endgroup$
    – moby1209
    Commented Jul 15, 2022 at 10:32
  • $\begingroup$ Why would you remove data in the first place? What are you trying to achieve with this? And how do you define "outliers"? $\endgroup$ Commented Jul 15, 2022 at 10:43
  • $\begingroup$ I define outliers as values that deviate more than three standard deviations from the mean. I want to remove them because I do not want the ML models to learn from unusual observations to improve the predictive out-of-sample performance. $\endgroup$
    – moby1209
    Commented Jul 15, 2022 at 10:48
  • 1
    $\begingroup$ I am saying that removing data simply because it causes problems is not a solution in general (UNLESS you have good reasons to remove them, such as erroneous measurements i.e. incorrect data). See also Is it appropriate to identify and remove outliers because they cause problems? $\endgroup$ Commented Jul 15, 2022 at 11:51

1 Answer 1



You should not remove outliers, because when you feed unseen data to the model that you have made, it will not be able to give good predictions for 'outliers in unseen data'. One way to make model generalize even when the data has more frequency of few categories is to sample your data - also called bootstrapping.

Bootstrapping will help model learn from more data.

  • $\begingroup$ Isn't resampling only necessary for classification problems? $\endgroup$
    – moby1209
    Commented Jul 15, 2022 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.