# How to tune the proportion of minority class after oversampling in imbalanced data with cross-validation? [duplicate]

I have an imbalanced data set and I want to balance them with SMOTENC with cross-validation. In order to determine the performance of the classifier on the original data, I will cross-validation as follows:

Here, feature X is divided into X_train and X_val.

scoring = {'accuracy' : make_scorer(accuracy_score),
'precision' : make_scorer(precision_score),
'recall' : make_scorer(recall_score),
'f1_score' : make_scorer(f1_score)}

model=RandomForestClassifier(n_estimators=50,random_state=10)

results = cross_validate(estimator=model, X=X, y=Y, cv=10, scoring=scoring)


Based on this code, I can find the performance of random forest on the original data.

However, I am a bit confused about how I should compare the oversampled data with the original data. I want to determine the best proportion of data and I use the following code. Should I split the original data to train and val and oversample train data and find the performance on val data? I am wondering if the results of these codes are comparable?

percentage = np.arange(0.20,0.5,0.02)
Measurements = np.zeros( (np.size(percentage),4) )

skf = StratifiedKFold(n_splits=10)

for train_index, test_index in skf.split(X, y):
X_train, X_val = X[train_index], X[test_index]
y_train, y_val = y[train_index], y[test_index]

counter = 0
for per in percentage:

smote_nc = SMOTENC(sampling_strategy = per, categorical_features=[0,1,2,3,4,6,8,9], random_state=0)
X_resampled, y_resampled = smote_nc.fit_resample(X_train, y_train)

X_resampled = pd.DataFrame(X_resampled, columns=X_val_train.columns)

after_X_resampled = encoding(X_resampled)

RF = RandomForestClassifier(n_estimators=50,random_state=10)
RF.fit(after_X_resampled, y_resampled)

prediction = RF.predict(X_val)

Measurements[counter,0] += accuracy_score(y_val, prediction)
Measurements[counter,1] += precision_score(y_val, prediction)
Measurements[counter,2] += recall_score(y_val, prediction)
Measurements[counter,3] += f1_score(y_val, prediction)

counter+=1

• In short - yes, since you have included SMOTE in your CV procedure and you apply it only to your training folds, you are OK. – desertnaut Mar 25 at 11:40