New answers tagged transformer
2
votes
Open-Source Large Language Models (LLM): Your experience and recommendation
You should check the Open LLM Leaderboard from Huggingface.
They maintain a ranking of open LLMs in different tasks. The ones finetuned to follow instructions are marked. When clicking on each model, ...
- 23.8k
1
vote
Accepted
What is the input to an encoder-decoder transformer in next word prediction task?
If you just want the model for doing next token prediction, then you would not use an encoder-decoder architecture. Instead, you would use a only the decoder, and feed the text you have to it to get ...
- 23.8k
1
vote
Accepted
How does Bert masked language modelling task make sense if half the time the next sentence is wrong context in the sequence passed through the encoder
First, note that the purpose of next sentence prediction objective is not to contribute to the contextual embeddings part, but to allow other downstream tasks like sentence classification and textual ...
- 23.8k
1
vote
Converting a Standard LSTM RNN over to a Transformer Model
With a bit of elastic net, dynamic gradient clipping and adjustments to the transformer model build training is progressing nicely now.
Here is the build that fixed it:
...
- 111
Top 50 recent answers are included