0
$\begingroup$

I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution :

enter image description here

And here are the classification results of my former classification (I took the numbers off the labels):

enter image description here.

Therefore I tried to improve classificationby downsampling the marjority classes

import os
from sklearn.utils import resample

# rebalance data
#df = resample_data(df)

if True:

    count_class_A, count_class_B,count_class_C, count_class_D,count_class_E, count_class_F, count_class_G = df.grade.value_counts()
    count_df = df.shape[0] 
    class_dict = {"A": count_class_A,"B" :count_class_B,"C": count_class_C,"D": count_class_D,"E": count_class_E, "F": count_class_F, "G": count_class_G}
    counts = [count_class_A, count_class_B,count_class_C, count_class_D,count_class_E, count_class_F, count_class_G]
    median = statistics.median(counts)


    for key in class_dict:
        if class_dict[key]>median:
            print(key)
            df[df.grade == key] = df[df.grade == key].sample(int(count_df/7), replace = False) 
                                             #replace=False,    # sample without replacement
                                             #n_samples=int(count_df/7),     # to match minority class
                                             #random_state=123)
# Divide the data set into training and test sets
x_train, x_test, y_train, y_test = split_data(df, APPLICANT_NUMERIC + CREDIT_NUMERIC,
                  APPLICANT_CATEGORICAL,
                  TARGET,
                  test_size = 0.2,
                  #row_limit = os.environ.get("sample"))
                  row_limit = 552160)

However it the results where catastrophic. The model accuracy and the model loss looked to have some issues :

enter image description here

And everything was classified in "A" on the test set.

$\endgroup$
  • $\begingroup$ You sure you are really down sampling the data? Does df shape changes? This line df[df.grade == key] = df[df.grade == key].sample(int(count_df/7), replace = False) seems weird to me. $\endgroup$ – yoav_aaa Sep 19 at 12:08
  • $\begingroup$ I'm not sure anymore. I also tried with df.loc[df.grade == key] = df.loc[df.grade == key].sample(. How would you have done it ? $\endgroup$ – IggyPass Sep 19 at 12:16
0
$\begingroup$

For the looks of it the sampling method you used isnt doing the trick. The way you use pandas slicing and assignment is a bit off.

Something like this might work:

dfs = []
for key in class_dict:

    if class_dict[key] > median:
        factor = int(count_df/7)
        key_df = df[df.grade == key].sample(factor, replace=False)
    else:
        key_df = df[df.grade == key]

    dfs.append(key_df)
new_train_df = pd.concat(dfs, sort=False)

While this might work, there are more straight forward ways down sampling the data(like setting the exact number of samples expected from each class).
This question has nice answers too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.