In short, Bidirectional Encoder Representations from Transformers (BERT) is not designed for decorder-related tasks.
I can't see how BERT makes predictions without using a decoder unit, which was a part of all models before it including transformers and standard RNNs. How are output predictions made in the BERT architecture without using a decoder? How does it do away with decoders completely?
If I undertand correctly, you are asking how BERT predicts its output. If you read the paper, BERT uses a technique called masked language modeling (MLM). During training, BERT randomly masks some of the tokens in a sentence and then tries to predict what the original word was. For example, in the sentence "I want to buy a [MASK]", BERT might be trained to predict that the masked word is "car" based on the context of the other words in the sentence.
The MLM task only requires an encoder because it involves encoding the input text into a fixed-length vector representation that can be used for downstream tasks such as sentiment analysis, question-answering, and language generation.
BERT also uses another technique called next sentence prediction. During training, BERT is given pairs of sentences and is asked to predict whether the second sentence follows logically from the first. This allows BERT to capture the relationships between sentences and to understand the broader context of a given text.
During inference, BERT uses its pre-trained weights to encode the input sentence and then passes the encoded representation through one or more neural network layers to generate the output. The exact details of how BERT generates its output depend on the specific task it is being used for, but in general, it uses a combination of the encoded input, attention mechanisms, and learned weights to produce the final output.
To put the question another way: what decoder can I use, along with BERT, to generate output text? If BERT only encodes, what library/tool can I use to decode from the embeddings?
I see you have edited your questions. In this newly added question, I can see that you have found that BERT only provides you with embeddings. You need other architectures like decorders along with these embeddings for your downstream tasks.
Here are some examples from the smiley face https://huggingface.co/learn/nlp-course/chapter1/6