1
$\begingroup$

I'm getting the below error while using class weights in the model.fit in tensorflow version 2.7.0

ValueError                                Traceback (most recent call last)
<ipython-input-21-27678ea5cbb9> in <module>
      1 # Fit the model
----> 2 history = model.fit(
      3     x={'input_ids': x['input_ids'], 'attention_mask': x['attention_mask']},
      4     y={'category': y_category },
      5     validation_split=0.2,

~\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb

~\Anaconda3\lib\site-packages\keras\engine\data_adapter.py in _class_weights_map_fn(*data)
   1434 
   1435     if tf.nest.is_nested(y):
-> 1436       raise ValueError(
   1437           "`class_weight` is only supported for Models with a single output.")
   1438 

ValueError: `class_weight` is only supported for Models with a single output.

My fit function is:

# Fit the model
history = model.fit(
    x={'input_ids': x['input_ids'], 'attention_mask': x['attention_mask']},
    y={'category ': y_category },
    validation_split=0.2,
    class_weight=d_class_weights,
    batch_size=6,
    epochs=10)

and I managed to get d_class_weights as:

from sklearn.utils.class_weight import compute_class_weight

y_category = np.argmax(y_category , axis=1)
class_weights = compute_class_weight('balanced', np.unique(y_category ), y_category )
d_class_weights = dict(enumerate(class_weights))
d_class_weights

If I run, without class_weights it's working fine, however the dataset is very unbalanced and hence this is needed.

Model:

Model: "BERT_MultiLabel_MultiClass"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 attention_mask (InputLayer)    [(None, 300)]        0           []                               
                                                                                                  
 input_ids (InputLayer)         [(None, 300)]        0           []                               
                                                                                                  
 bert (TFBertMainLayer)         TFBaseModelOutputWi  109482240   ['attention_mask[0][0]',         
                                thPooling(last_hidd               'input_ids[0][0]']              
                                en_state=(None, 300                                               
                                , 768),                                                           
                                 pooler_output=(Non                                               
                                e, 768),                                                          
                                 hidden_states=None                                               
                                , attentions=None)                                                
                                                                                                  
 pooled_output (Dropout)        (None, 768)          0           ['bert[0][1]']                   
                                                                                                  
     category (Dense)           (None, 32)           24608       ['pooled_output[0][0]']          
                                                                                                  
==================================================================================================
Total params: 109,506,848
Trainable params: 109,506,848
Non-trainable params: 0
__________________________________________________________________________________________________
$\endgroup$
2
  • $\begingroup$ What type of model are you trying to fit on your data? $\endgroup$
    – Oxbowerce
    Feb 16, 2022 at 15:03
  • $\begingroup$ @Oxbowerce i have added the model summary in description. using BERT $\endgroup$
    – nikviz
    Feb 16, 2022 at 15:07

1 Answer 1

1
$\begingroup$

This is a known tensorflow bug. There's no solution to it but using a custom loss function.

$\endgroup$
2
  • $\begingroup$ Thanks for this link.. I managed to work around this error by downgrading tensorflow from 2.8.0 to 2.1.0 which meant I also had to downgrade python to 3.7 and transformers package to to 3.1.0 $\endgroup$
    – nikviz
    Feb 17, 2022 at 14:56
  • $\begingroup$ I'm glad you could have it solved. Please consider upvoting the answer and marking it as correct. $\endgroup$
    – noe
    Feb 17, 2022 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.