Using class weights with validation data in Keras

Question

I am using validation data for hyperparameter optimization and am trying to use class weights. For model.fit(), there is an option to pass class weights (the argument is class_weight) to the function for test data, but I would like to do the same to validation data.

Another user had the same question a while back, but no satisfactory answers were given:

Class weighting during validation in Keras

When passing validation data as an argument to model.fit(), the validation data can take the form of a tuple: (x_val, y_val, val_sample_weights), but in the Keras documentation I am not seeing any way to use class weights instead of sample weights.

Does anyone have any idea on how to use class weights with validation data in Keras?

Here is some info that might be helpful:

From the Keras documentation, description of the class_weight argument:

Dictionary mapping class indices (integers) to a weight (float) value, used for weighting the loss function (during training only). This can be useful to tell the model to "pay more attention" to samples from an under-represented class.

Here is a notebook from Francois Chollet which shows an example of using class_weight: https://colab.research.google.com/drive/1xL2jSdY-MGlN60gGuSH_L30P7kxxwUfM#scrollTo=REP1KlrSEx-m

Thanks for any help!

Answer 1

Class weights make sense only in the context of a loss function. When you validate your model you are making predictions and comparing to ground truth using a metric - but in that phase you aren't propagating back any changes, so weights are useless.

Answer 2

I think it is a pity that Keras does not comfortably allow class weights on the validation set. However, you can translate class weights to sample weights and plug those into the last element of the tuple: (x_val, y_val, val_sample_weights).

In the binary classification example you provided, the translation could be done via:

val_sample_weights = val_targets*class_weight[1] + (1-val_targets)*class_weight[0].

Then you should have an array with the same length as the validation set, containing the desired class weight for each data point.