everyone hope you all are good. I have a hard time with outliers First i found some outliers in train set then i capped them and did the same with test set. So when i checked my rsme root mean squarred error was high but when i capped train set outlier and did not capped test set outliers then my rsme was decresed . I donot know what should i do should i cappe both train and test set outliers or i should only cape the train set outlier and donot do anything with test set outliers Please i am new to data science if you guys please help me
6
-
$\begingroup$ Would you like to share more context about your dataset and the outliers? It really depends on the nature of the dataset whether you should drop the outlier or not. For example - Your dataset contains a dataset of pictures of dogs but it also includes a picture of cat ; then this picture of cat would be considered an outlier and would have to be dropped in both the datasets because that is an incorrect data point which incurred in data collection process. $\endgroup$– KritiNov 6 at 17:26
-
$\begingroup$ Thanks alot now i understand outliers dropping reasons. In my dataset outliers are in seeds quantity for example there is one column called seeds planted per pit so in every cultivated land in each pit 2 or 3 or 5 seeds are planted but there are pits where 500 300 150 seeds are planted so i think that these are outliers. So what to do with these numbers $\endgroup$– Shoaib AhmedNov 9 at 14:41
-
$\begingroup$ Yes, you are right. IMO, you should be dropping those 500,300,150 values entirely from the dataset as it could be a data entry error. Either you re-measure your data point and correct it or you remove it completely from the data set. $\endgroup$– KritiNov 9 at 16:06
-
$\begingroup$ Please consider upvoting comments or marking the answer as correct, if you find anything helpful $\endgroup$– KritiNov 9 at 16:30
-
$\begingroup$ I have upvoted but it says you cannot upvote your own post $\endgroup$– Shoaib AhmedNov 10 at 14:48
|
Show 1 more comment