1
$\begingroup$

I am working on very imbalanced dataset, I used SMOTEENN (SMOTE+ENN) to rebalance it, the following test is made using Random Forest Classifier :

My train and Test score before using SMOTEENN:

print('Train Score: ', rf_clf.score(x_train, y_train))
print('Test Score: ', rf_clf.score(x_test, y_test))
Train Score: 0.92
Test Score: 0.91

After using SMOTEEN :

print('Train Score: ', rf_clf.score(x_train, y_train))
print('Test Score: ', rf_clf.score(x_test, y_test))
Train Score: 0.49
Test Score: 0.85

Edit

x_train,x_test,y_train,y_test=train_test_split(feats,targ,test_size=0.3,random_state=47)

scaler = MinMaxScaler()
scaler_x_train = scaler.fit_transform(x_train)
scaler_x_test = scaler.transform(x_test)
X = scaler_x_train
y = y_train.values

from imblearn.over_sampling import SMOTE
from imblearn.under_sampling import EditedNearestNeighbours
from imblearn.combine import SMOTEENN
   
oversample = SMOTEENN(random_state=101,smote=SMOTE(),enn=EditedNearestNeighbours(sampling_strategy='majority'))
start = time.time()
X, y = oversample.fit_resample(X, y)
stop = time.time()
print(f"Training time: {stop - start}s")

rf_model = RandomForestClassifier(n_estimators=200, class_weight='balanced', criterion='entropy', random_state= 0, verbose= 1, max_depth=2)
rf_mod = OneVsRestClassifier(rf_model)
rf_mod.fit(X, y)
$\endgroup$
4
  • $\begingroup$ it might depend on train_test_split and how this was done before and after SMOTENN. Try different splits $\endgroup$
    – Nikos M.
    Jun 11, 2021 at 15:27
  • $\begingroup$ test_size=0.3, and the score before SMOTTEEN was good : Train score: 0.92 and Test score: 0.91 $\endgroup$
    – Mimi
    Jun 11, 2021 at 16:25
  • $\begingroup$ how do you use SMOTEENN and how do you train test split? please explain these steps in detail $\endgroup$
    – Nikos M.
    Jun 11, 2021 at 16:28
  • $\begingroup$ @NikosM. I edited it $\endgroup$
    – Mimi
    Jun 11, 2021 at 16:42

1 Answer 1

0
$\begingroup$

You are probably not applying the same resampling technique to test dataset. If you put the logic into a imbalanced-learn's Pipeline, the appropriate resampling will be automatically handled for you.

$\endgroup$
2
  • 1
    $\begingroup$ I used SMOTEENN only for Train set. See the discussion in link : datascience.stackexchange.com/questions/15630/… $\endgroup$
    – Mimi
    Jun 11, 2021 at 18:41
  • $\begingroup$ I too think splitting and then resampling only on train set should be performed (agree with the linked post). $\endgroup$
    – Nikos M.
    Jun 12, 2021 at 9:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.