3
$\begingroup$

Is it possible to use class_weights with a one-hot encoding?

I've tried sparse_categorical_crossentropy and, for some reason, it's significantly worse than my classic categorical_crossentropy with one hot encoding model.

This is how I'm calculating the class_weights with sparse:

unique_class_weights = np.unique(labels)
class_weights = class_weight.compute_class_weight('balanced', unique_class_weights, labels)
class_weights_dict = { unique_class_weights[i]: w for i,w in enumerate(class_weights) }

Training like:

full_model.compile(loss='sparse_categorical_crossentropy',
                   optimizer='rmsprop',
                   metrics=['accuracy'])


full_model.fit_generator(line_generator(data_train, labels_train),
                         validation_data=line_generator(data_test, labels_test),
                         validation_steps=1,
                         steps_per_epoch=len(data)/GENERATOR_BATCH_SIZE,
                         class_weight=class_weights_dict,
                         epochs=1)
Improve this question
$\endgroup$
3
$\begingroup$

I'm not allowed to comment, but have you tried using the numpy array that you get from class_weight.compute_class_weight(), rather than converting it to a dict? I've always skipped that part, and in your case I would say class_weight=class_weights. Sorry if I'm suggesting something you already ruled out. Good luck.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I have tried that. Unfortunately, I couldn't find an official reference that it's a correct approach. On Github the official examples are using a dict, and there are some issues where it's been said that one hot encoding and class weights don't work. So I'm confused about this to be honest. I don't know. $\endgroup$
    – MB.
    Nov 14 '17 at 6:15
  • $\begingroup$ Good point. It looks like the code called by fit_generator actually checks for a dict, then converts it to a numpy array. Hope you find your answer soon. $\endgroup$
    – Hunter McGushion
    Nov 14 '17 at 6:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.