New answers tagged class-imbalance
0
Intuitively, it seems like an imbalanced dataset to have ~75/25 ratio of class labels.
If you want to take a look at it theoretically, you can do a hypothesis test. For a sample size of 8161, you can assume that the dataset is 50/50 as null hypothesis and then compute the probability that a number extreme as 6008 or more of them belong to one class as p-...
0
class_weight does not influence the composition of the batches. Instead, it applies a weight to the loss function that depends on the weight of the class.
3
You must apply SMOTE after splitting into training and test, not before. Doing SMOTE before is bogus and defeats the purpose of having a separate test set.
At a really crude level, SMOTE essentially duplicates some samples (this is a simplification, but it will give you a reasonable intuition). If you duplicate every sample ten times, and then split into ...
6
Essentially applying SMOTE makes the job easier for the model: SMOTE generates artificial instances which tend to have the same properties as each other, so it's easier for the model to capture their patterns. However these instances are rarely a good representative sample for the minority class, so there's a higher risk that the model overfits.
Of course if ...
2
Training on the resampled data causes the model to assign roughly the same proportion of positive and negative labels, especially in difficult cases where it doesn't have clear enough indications from the features.
When testing on the test set with the original distribution (this is the correct method of course), the model applies what it learned: for "...
1
predict_proba method will return a numpy array of shape (n_samples,2) with the probability of Y == 1 and Y == 0 but you need to pass only the probability of Y == 1 for roc calculation so:
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
clf = LogisticRegression(solver="liblinear&...
0
That is an empirical question that could be answered through hold-out datasets. Create the different scenarios and see which one the model performs better in.
1
from imblearn.metrics import geometric_mean_score
Docs are here.
2
Your assessment is right. You must first determine the data distribution in real-time (production) and only after that proceed with train_set, test_set and validation_set creation with the same distribution. And subsequently work on model training and setting the class weightages if required.
Why:
Any metrics upon which you evaluate your model are basically
...
Top 50 recent answers are included