I'm working on a project, I finished data preprocessing, and I found an article where it says that feature scaling and feature selection should be done after splitting data, some other articles say it should be done before. I also found an article that suggests splitting data first and then preprocessing it. I'm a bit confused, does anyone know the real answer? Thank you.
1
-
$\begingroup$ Hi @bihu, Do data pre-processing before splitting, this will keep experiment pipeline streamlined. You can do it after splitting too, but then you will have to repeat the pre-processing process. While doing pre-processing before data split, ensure no leakage of target occurs. $\endgroup$– DataFramedCommented Oct 17, 2022 at 13:00
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In general, test data insights should not affect our decisions, and test data should not leak into model training process.
What happens when we perform certain actions before splitting?
Exploration: if we split the sets ourselves, we assume the distributions having no significant differences, so performing exploratory analysis on the full data is acceptable.
Feature selection: once again, if we assume the distributions to be roughly the same, stats like mutual information or variance inflation factor should also remain roughly the same. I'd stick to selection using the train set only just to be sure.
Imputing missing values: filling with a constant should create no leakage. Strategies like filling with mean values result in a leakage (albeit a minor one, if it's a school project, reviewers are known to often turn a blind eye on that).
Dropping outliers: if we expect the real input data to be filtered the same way, it can be done before splitting. Otherwise it's just cheating (we exclude observations that are hard to predict from all sets).
Feature encoding: depends on the encoding strategy. OHE shouldn't result in a leakage, target encoding would make a huge leak.
Feature engineering: if we operate within a single observation (like, adding two features' ratio), there's no leak. If it's based on other observations somehow, it's a leak (this applies to most scaling methods just as well). If it considers target too (e.g. SymbolicTransformer()), the leak becomes really huge.
$\begingroup$
$\endgroup$
Strictly you should not act as if you know things in your test set.
If you want to perform min-max scaling (for instance) you should fit the min-max scaler to your training set. Then use this to transform your training and test set.
Perform feature selection in the same way.
(I say training and test because the validation will be part of the training)
