I have the following issue with the BERT transformer in python:

When I feed to BertModel().from_pretrained('bert-base-cased') an input obtained from BertTokenizer.from_pretrained('bert-base-cased') that contains a value bigger than 29000 I receive an error message saying that one of the indexes is out of range.

What I tried to do is to use BertConfig and set a vocab_size higher than 29000:

configuration = BertConfig(vocab_size=30_522)
bert = BertModel(config=configuration).from_pretrained('bert-base-cased')

Do you know where I can see which value of vocab_size is actually being used? Do you know how I can solve my issue?

enter image description here



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.