Consider a binary classification scenario whereby the True class (5%) is severely outbalanced to the False class (95%). My data set contains numeric data. I am using SKLearn and trying some different algorithms such as Gradient Boosting Classifier (GCB), Random Forest (RDC) and Support Vector Classifier (SVC). Due to the unbalanced aspect, I am using "sample_weight" in all the methods (fit, score, confusion_matrix, etc) and populating it with the below weight array, whereby, True values are given a value of 20 and False values are given a value of 1.
sample_weight = np.array([20 if i == 1 else 1 for i in y_test])
This is supposed to "balance" the classification. Is this a correct approach first of all? And what are the implications? Increasing the sample size with real data is impossible. Overfitting would be another potential option, but I thought to start by the "sample_weight" approach due to the significant discrepancy between classes.
Now consider these results:
GCB - Accuracy 71%, Precision 74%, Recall 69%, F1-Score 71%
RFC - Accuracy 67%, Precision 82%, Recall 47%, F1-Score 60%
SiVC - Accuracy 63%, Precision 74%, Recall 45%, F1-Score 56%
GCB outperforms the other algorithms in Accuracy. RFC in Precision. Recall is very bad in both RFC and SVC, and the least value in the 3 cases.
My next question is mainly with regards to recall. The False negatives are considerable - as shown by the recall (in the SVC case, 55% of actual True values were predicted as False); is this because in the original data set there are more False values?
Where does the sample weighting come in in all of this? Isn't the sample weighting supposed to mitigate? Or does the class imbalance still affect the classification regardless of the weighting; in some cases more than others (case in point RFC and SVC more than GCB)?