Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
From what i have learned, people say that it is more correct if i preprocess data after splitting it to train/test dataset. My questions are
1.Does it mean we detect flaws of the data + preprocess it after the splitting? if yes/no, why?
2.Is it okay if i detect flaws of the data before splitting it and then preprocess the flaws after splitting it? if it is/ no it is not, why?
The goals of all these methodological guidelines is to avoid data leakage.
Example: let's imagine we want to classify short messages (e.g. tweets). When inspecting the data we find various kinds of smileys: :-), :|, :-/... At preprocessing stage we replace all smileys found in the data with a special token like <smiley> (or something more specific).
<smiley>.In the first case there is data leakage: we fixed some issues in the test set even though this wouldn't have been possible with actual fresh data (here the variants of smiley that were not seen in the training set). In the second case the test set is "imperfect", i.e. it's exactly as if it was made of "fresh" unseen data, therefore the evaluation will be more realistic.
This example shows why it's always safer to separate the data first, design the preprocessing steps on the training data, then apply exactly the exact same preprocessing steps to the test data.
In practice there can be cases where it's more convenient to apply some general preprocessing to the whole data. The decision depends on the task and the data: sometimes the risk of data leakage is so small that it can be neglected. However it's crucial to keep in mind that even the design of the preprocessing can be a source of data leakage.
