# Can BERT be used for predicting words?

I have a question regarding the pre-training section (in particular, the Masked Language Model).

In the example Let's stick to improvisation in this skit, by masking the word improvisation, after applying BERT (followed by the FFNN and Softmax), by looking at the probabilities of each of the possible English words, we are able to correctly predict that the masked word was improvisation.

Is it possible to actually play with this by using my own examples? I was wondering if I can input a sentence in a different language (from the multilingual model) and have a sorted list of the most probable words that were masked in the original sentence. If it's possible, what needs to be tweaked?

Any help would be greatly appreciated.

pip install transformers
from transformers import pipeline

• Thank you, that works! Just one mention: if anyone tries this and gets the message: Exception: Impossible to guess which tokenizer to use. Please provided a PretrainedTokenizer class or a path/identifier to a pretrained tokenizer. they should specify the model in the pipeline (e.g. "bert-base-cased") – moz_szt Apr 17 at 9:56