I am working on a logistic regression case with an imbalance in the target variable. To fix this I am using SMOTE (Synthetic Minority Oversampling Technique), but each time I run my regression model, I get different numbers in my confusion matrix. I have set random_state parameters while invoking SMOTE as well as Logistic Regression still to no avail. Even my features are the same in each iteration. I was able to get the best value for recall as 0.81 and AUC value as 0.916 once but they are not coming anymore. On some occasions, the value of False Positives and False Negatives shoots up very much indicating that the classifier is very bad.
Please guide what I am doing wrong here, below is the code snippet.
# Feature Selection features = [ 'FEMALE','MALE','SINGLE','UNDER_WEIGHT','OBESE','PROFESSION_ANYS', 'PROFESSION_PROF_UNKNOWN'] # Set X and Y Variables X5 = dataframe[features] # Target variable Y5 = dataframe['PURCHASE'] # Splitting using SMOTE from imblearn.over_sampling import SMOTE os = SMOTE(random_state = 4) X5_train, X5_test, Y5_train, Y5_test = train_test_split(X5,Y5, test_size=0.20) os_data_X5,os_data_Y5 = os.fit_sample(X5_train, Y5_train) columns = X5_train.columns os_data_X5 = pd.DataFrame(data = os_data_X5, columns = columns ) os_data_Y5 = pd.DataFrame(data = os_data_Y5, columns = ['PURCHASE']) # Instantiate Logistic Regression model (using the default parameters) logreg_5 = LogisticRegression(random_state = 4, penalty='l1', class_weight = 'balanced') # Fit the model with train data logreg_5.fit(os_data_X5,os_data_Y5) # Make predictions on test data set Y5_pred = logreg_5.predict(X5_test) # Make Confusion Matrix to compare results against actual values cnf_matrix = metrics.confusion_matrix(Y5_test, Y5_pred) cnf_matrix