0
$\begingroup$

I am new to NLP and would like to build a BERT model for sentiment analysis so I am following this tutorial.

However, I am getting the error below:

F.softmax(model(input_ids, attention_mask), dim = 1)

When I would like to execute this cell I get the error:

 dropout(): argument 'input' (position 1) must be Tensor, not str
$\endgroup$
1
$\begingroup$

How to obtain the same behavior as v3.x in v4.x In order to obtain the same behavior as version v3.x, you should install sentencepiece additionally:

In version v3.x:

pip install transformers

to obtain the same in version v4.x:

pip install transformers[sentencepiece] or

pip install transformers sentencepiece

$\endgroup$
1
$\begingroup$

Which transformer version are you using? I had to pin mine to transformer == 3.5.1 to mitigate that problem, when the hugging face team updated their transformer to 4.0 things started to break.

Hope it helps

$\endgroup$
0
$\begingroup$

The transformers library uses complex output objects instead of plain tuples as return type since one of the updates after 3.5.1.:

from transformers import BertModel, BertTokenizer

t = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')

o = t.encode_plus('this is a sample sentence', return_tensors='pt')

mo= model(**o)
print(type(mo))
print(mo.keys())

Output:

<class 'transformers.modeling_outputs.BaseModelOutputWithPoolingAndCrossAttentions'>
odict_keys(['last_hidden_state', 'pooler_output'])

You have to change your Sentimentclassifier to either return to the previous behavior by specifying return_dict=False or use the BaseModelOutputWithPoolingAndCrossAttentions (recommended):

class SentimentClassifier(nn.Module):
  def __init__(self, n_classes):
    super(SentimentClassifier, self).__init__()
    self.bert = BertModel.from_pretrained(PRE_TRAINED_MODEL_NAME)
    self.drop = nn.Dropout(p=0.3)
    self.out = nn.Linear(self.bert.config.hidden_size, n_classes)

  def forward(self, input_ids, attention_mask):
    bertOutput = self.bert(
      input_ids=input_ids,
      attention_mask=attention_mask
    )
    output = self.drop(bertOutput['pooler_output'])

    return self.out(output)

Please have a look at this stackoverflow post in case you do not want to use my recommended solution.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.