1
$\begingroup$

I have a dataset with around 4.7K focused on binary classification. Class proportion is 33:67. meaning Label 1 is 1558 (33%) and Label 0 is 3154 (67%) of my dataset.

Is my dataset imbalanced? some people say it is not bad

My objective is to increase the F1-score only. I set the class_weight=balanced in my parameters and scoring=f1 during CV as shown below.

svm=SVC(random_state=42)
svm_cv=GridSearchCV(svm,param_grid,cv=5,scoring='f1')
svm_cv.fit(X_train_std,y_train)

Can you let me know through a code sample on how I can increase the weightage to minority class? if that is any different from choosing balanced paramter

Currently my results are like as follows

enter image description here

I understand AUC for few algo is above 80 but I believe F1-score is more important for a imbalanced class problems like mine.

Can you help? I tried Oversampling minority class but not much improvement.

Increasing features doesn't take me to 80% F1-score

Improve this question
$\endgroup$
1
$\begingroup$

I would say you data in not imbalanced. 33:67 is not a bad ratio but Try using under sampling of majority class. As another option you can try difference algorithms like randomforest. You can also try boosting.

Improve this answer
$\endgroup$
  • $\begingroup$ Welcome to SO. Sure. will try your suggestion. Upvoted for the help $\endgroup$ – The Great Dec 19 '19 at 6:25
  • $\begingroup$ btw i already tried different algorithms as you can see from screenshot $\endgroup$ – The Great Dec 19 '19 at 6:25
  • $\begingroup$ How many feature are there in your data? Have you tried confusion matrix? $\endgroup$ – shahid hamdam Dec 19 '19 at 6:30
  • $\begingroup$ Yes, but confusion matrix just gives the TPR.FPR,FNR etc. Yes, I did create a confusion matrix $\endgroup$ – The Great Dec 19 '19 at 6:31
  • $\begingroup$ I have 60 features in my data $\endgroup$ – The Great Dec 19 '19 at 6:31
1
$\begingroup$

I call imbalanced dataset is when you have a ratio of 90|10 at least. Your problem is not imbalanced.

F1 score is not a Loss Function but a metric. In your GridsearchCV you are minimising another loss function and then selecting in your folds the best F1 metric. It is important to understand these concepts.

If you want to apply Oversample/Undersample techniques you can use the following library. (Even if you don't need it)

https://pypi.org/project/imbalanced-learn/

If you want to improve your score you could try with another algorithm such as Gradient Boosting with its different implementations XGB, LightGBM, catboost ...

Improve this answer
$\endgroup$
  • $\begingroup$ thanks for the response. Upvoted $\endgroup$ – The Great Dec 19 '19 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.