X = all features from dataset

y = all target from dataset

X_train = features that already using train_test_split approach

y_train = target that already using train_test split approach

So my question is which one should I choose if I would like to do hyperparameter tuning? I have imbalanced data. In this case I would like to make pipeline that contains smote and the algorithm. I read someone who said that you should do oversampling on each fold of cross validation. Assuming when I am using randomized search CV --> that also have cross validate I am decided to run smote in pipeline. But I am unsure which data should I fit after I run the code.

fit(X,y) or fit(X_train, y_train)

2 Answers 2


The recommended approach is to use cross validation on the training dataset (X_train, y_train) for hyperparameter tunning and oversampling on each fold of cross validation.

The code would something like this:

from imblearn.over_sampling import SMOTE
from imblearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, RandomizedSearchCV, StratifiedKFold

X_train, y_train, X_test, y_test = train_test_split(X, y)

pipeline = Pipeline([("smote", SMOTE()), ("rf", RandomForestClassifier())])

kf = StratifiedKFold()

rscv = RandomizedSearchCV(estimator=pipeline, cv=kf)
rscv.fit(X_train, y_train)
  • $\begingroup$ thank you for your answer that help me to understand. So when i'm applying smote into pipeline, it applying oversampling in each fold (because randomized search CV using cross validation technique). But can i ask you one more question regarding oversampling? datascience.stackexchange.com/questions/115218/… in that case i dont want to tuning hyperparameter.. I just want to evaluate my model so i think it doesnt matter when i'm applying smote before or applying it in pipeine. $\endgroup$ Commented Oct 15, 2022 at 3:21


  • Certainly not the whole dataset (X,y) because this would cause data leakage and invalidate the evaluation.
  • The training set (X_train, y_train) should be used only for training.

The solution is to split the training set into a training set and validation set. Equivalently you can use cross-validation in the full training set, since the CV process would take care of splitting the data.

If interested, this is another explanation about using a validation set for parameter tuning.

If you resample (I don't recommend it, at least not without a good reason), you should never do it on the validation set or the test set.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.