everyone hope you all are good. I have a hard time with outliers First i found some outliers in train set then i capped them and did the same with test set. So when i checked my rsme root mean squarred error was high but when i capped train set outlier and did not capped test set outliers then my rsme was decresed . I donot know what should i do should i cappe both train and test set outliers or i should only cape the train set outlier and donot do anything with test set outliers Please i am new to data science if you guys please help me

  • $\begingroup$ Would you like to share more context about your dataset and the outliers? It really depends on the nature of the dataset whether you should drop the outlier or not. For example - Your dataset contains a dataset of pictures of dogs but it also includes a picture of cat ; then this picture of cat would be considered an outlier and would have to be dropped in both the datasets because that is an incorrect data point which incurred in data collection process. $\endgroup$
    – Kriti
    Commented Nov 6, 2023 at 17:26
  • $\begingroup$ Thanks alot now i understand outliers dropping reasons. In my dataset outliers are in seeds quantity for example there is one column called seeds planted per pit so in every cultivated land in each pit 2 or 3 or 5 seeds are planted but there are pits where 500 300 150 seeds are planted so i think that these are outliers. So what to do with these numbers $\endgroup$ Commented Nov 9, 2023 at 14:41
  • $\begingroup$ Yes, you are right. IMO, you should be dropping those 500,300,150 values entirely from the dataset as it could be a data entry error. Either you re-measure your data point and correct it or you remove it completely from the data set. $\endgroup$
    – Kriti
    Commented Nov 9, 2023 at 16:06
  • $\begingroup$ Please consider upvoting comments or marking the answer as correct, if you find anything helpful $\endgroup$
    – Kriti
    Commented Nov 9, 2023 at 16:30
  • $\begingroup$ I have upvoted but it says you cannot upvote your own post $\endgroup$ Commented Nov 10, 2023 at 14:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.