New answers tagged

2 votes

Open-Source Large Language Models (LLM): Your experience and recommendation

You should check the Open LLM Leaderboard from Huggingface. They maintain a ranking of open LLMs in different tasks. The ones finetuned to follow instructions are marked. When clicking on each model, ...
noe's user avatar
  • 23.8k
1 vote
Accepted

What is the input to an encoder-decoder transformer in next word prediction task?

If you just want the model for doing next token prediction, then you would not use an encoder-decoder architecture. Instead, you would use a only the decoder, and feed the text you have to it to get ...
noe's user avatar
  • 23.8k
1 vote
Accepted

How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder

First, note that the purpose of next sentence prediction objective is not to contribute to the contextual embeddings part, but to allow other downstream tasks like sentence classification and textual ...
noe's user avatar
  • 23.8k
1 vote

Converting a Standard LSTM RNN over to a Transformer Model

With a bit of elastic net, dynamic gradient clipping and adjustments to the transformer model build training is progressing nicely now. Here is the build that fixed it: ...
Ted Wilmont's user avatar

Top 50 recent answers are included