0
$\begingroup$

Let's say I've found some outliers in a column in my dataset and have decided to remove them.

Should I do this before or after I split the dataset into train/test sets?

$\endgroup$
1
$\begingroup$

If you decided to remove outliers. Please remove them before the split(even not only before a split, it's better to do the entire analysis(stat-testing, visualization) again after removing them, you may find interesting things by doing this).

If you remove outliers in only any one of train/test set it will create more problems. (EX: An outlier in train set may not be an outlier in combined/full set, also the model will have high variance if you do so)

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ @vekatesh U are right absolutely. One should always wipe features in prior if it founds. $\endgroup$ – Gaurav Koradiya Jun 9 at 6:55
0
$\begingroup$

I think two case is not different too much. (Eventually the all outlier is deleted)

But the proportion of the train/test set(e.x : 7:3) may different if you remove the outlier after the split.

So I recommend remove outlier before split if you could.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.