I am dealing with a highly unbalanced data, so I used the SMOTE algorithm to resample the dataset.
After SMOTE resampling, I splitted the resampled dataset to training/testing sets, using the training set to build a model and testing set to evaluate the model.
However, I am worried about that some data points in the testing sets might actually jittered from data points in the training set (i.e. the information is leaking from the training set to testing set), so the testing set is not really a clean set for testing.
Does anyone have similar experience? Does the information really leak from training to testing? Or the SMOTE algorithm actually took care of it and we don't have to worry about it?