You can also take a random word generator and use it to "generate text". Does it generate text? Well, yes, but no. The same applies to BERT.
The problem with using BERT for text generation is that it is not meant for that. BERT is a Transformer encoder. It is meant to receive as input a whole piece of text, and it will generate one output per input token (trying to guess any tokens in the input masked with the
[MASK] token) plus a sentence-level output (the output at the first position, which receives the
[CLS] token). Therefore, BERT is not trained to receive only the previous tokens and generate the next one.
So to answer your question: of course, you can use BERT to generate text by placing a
[MASK] token at the end of a sequence of tokens and using its prediction to autoregressively build more text, but don't expect a good output, at least not on the level of a "causal" language model like GPT-2 or GPT-3, which are Transformer decoders trained specifically to generate the next token based on the previous ones.